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Building a Framework for a Next-Generation English 
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Currently, states are moving forward with developing or adopting new English language proficiency (ELP) assessments aligned with the 
new academic standards in order to meet accountability requirements. In this transition, it is essential to identify areas of improvement 
for current ELP assessment practices and discuss ways to implement effective ELP assessments in order to better serve the needs of 
English language learner (ELL) students. This article offers a conceptual framework and guiding principles to consider in designing 
and developing next-generation ELP assessment systems for K-12 ELL students. In particular, the article calls for a need to create an 
ELP assessment system where the standards, a set of coherent assessments, and instruction are all systematically linked to maximize 
assessment results and thus aid in ELL students’ academic success. The importance of defining an overarching construct to govern 
various assessments and the use of evidence-centered-design principles for the development of an assessment system are also discussed. 
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English language learner (ELL) students are the fastest growing subpopulation in K-12 public schools in the United 
States. Over the 10-year period between the 1998- 1999 and 2008-2009 school years, the enrollment of ELL students in 
the nation’s public schools grew by over 51%, while growth in the total K-12 population was just over 7% (Office of English 
Language Acquisition [OELA], 2011). In the 2010-2011 school year, the number of students participating in programs 
for ELLs in Grades K-12 was nearly 4.7 million, representing approximately 10% of all public school students (Snyder 
& Dillow, 2013). The majority of ELL students reside in California, Texas, Florida, New York, and Illinois, accounting for 
large proportions of the overall K-12 populations in these states. In California, for example, approximately 1.6 million, 
or 25%, of K-12 students are classified as ELLs (Gandara, Maxwell-Jolly, & Driscoll, 2005). Growth in the ELL student 
population is occurring not only in the aforementioned states, but also in states that have not historically experienced 
high levels of immigration. To name a few, Colorado, Georgia, Indiana, and North Carolina have yielded over 200% of 
the ELL population growth over the past 10 years (OELA, 2011). 

ELL students are typically considered to be an underserved population in education and have received heightened 
attention due to their low academic performance in general. Much research has raised serious concerns about fairness 
and equity in opportunities to learn for ELL students. As ELL students deal with the dual challenge of learning academic 
content while developing their English language proficiency (ELP), an essential task to address these students’ immediate 
needs is helping them develop appropriate ELP to have better access to content learning and achieve academic success. 

In the face of this challenge, a promising development is the emergence of a great body of literature outlining effective, 
research-based strategies that promise to improve ELL education (August & Shanahan, 2006; Bailey, 2007; Calderon, 2007; 
Francis, Rivera, Lesaux, Kieffer, & Rivera, 2006; Goldenberg, 2008; Hakuta, 2011; Heritage, 2008; Snow, 2002; Valdes & 
Wong Fillmore, 2011). One particularly well-recognized strategy is the development of quality assessments and the effec¬ 
tive use of assessment results (Working Group on ELL Policy, 2011). Sound assessments can guide and support instruction, 
demonstrating concrete examples of expected learning outcomes (Herman, Osmundson, & Dietel, 2010). Students can 
also increase self-learning skills while engaging in high-quality assessment tasks. 

However, current assessment practices, which place a heavy emphasis on accountability testing, have clear limitations. 
The extent to which current ELP assessments provide useful information for ELL teachers also remains in question. As 
there is a prevailing concern that accountability tests provide little information to help guide teachers in their instruction, 
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attention has increasingly been paid to the need for assessments focused on improving teaching and learning (e.g., for¬ 
mative assessment). One of the criticisms of current state accountability tests is that they embody a product-oriented 
emphasis, neglecting to gather evidence of students’ learning processes. Teachers have also complained that state assess¬ 
ment results provide little information or insight into what or how to instruct in order to address students’ areas of need 
(Short & Fitzsimmons, 2007; Wolf, 2011). 

To address the critical need to support ELL students’ development of ELP, this article focuses on the development of 
quality ELP assessments. In particular, the goal of this article is to propose a conceptual framework and guiding principles 
to consider in designing and developing an improved ELP assessment system. In recognition of an unbalanced emphasis 
on the accountability ELP assessments employed in the current practice, we propose a framework for the creation of an 
integrated assessment system comprising a series of assessments measuring students’ language proficiency in a coherent 
manner. The intent of the proposed assessment system is to provide meaningful and actionable information to help stu¬ 
dents advance to higher levels of ELP and achieve academic success. This article places particular emphasis on elaborating 
on the overall ELP construct that underlines a series of assessments. We expect the framework and guiding principles laid 
out in this article to be informative to those who conceptualize, develop, and implement new ELP assessments. There¬ 
fore, our primary intended audience includes all decision makers involved in the implementation of assessments for ELL 
students, as well as actual assessment developers. 

In the following sections, we first briefly describe the current status of ELP assessments to understand the background 
context for our proposed ELP assessment system. Then, we illustrate the ELP assessment system, its major components, 
and the conception of the overarching ELP construct that governs the proposed assessment system. Additionally, we 
provide a brief overview of evidence-centered design (ECD) and other principles to consider for future assessment devel¬ 
opment based on this conceptual framework and construct. Finally, we suggest a list of studies to validate and enhance 
the ELP assessment system. 


Current Status of English Language Proficiency Assessment Practices 

The No Child Left Behind Act of2001 (NCLB, 2002) has made a significant impact on the current status of ELP assessment 
and its uses. The act stipulates that all states be held accountable for ELL students’ progress in ELP attainment to support 
meaningful participation in academic contexts. With this mandate, annual summative ELP assessments have been devel¬ 
oped and implemented in all schools. These ELP assessments, developed for accountability purposes, inform relatively 
high-stakes decisions for individual ELL students, including use as a primary criterion for determinations on readiness 
to exit ELL status. However, schools were also in need of assessments to aid in the identification of ELL students upon 
their initial enrollment in school. While each state implemented a state-wide ELP assessment for annual accountability 
purposes, a wide range of assessments have been utilized for the initial measurement of students’ ELP across schools 
(National Research Council, 2011; Wolf, Farnsworth, & Herman, 2008). 

While NCLB has had a positive influence insofar as it has emphasized the importance of ELP assessments and supported 
ELL students’ ELP attainment, current mainstream ELP assessment practices have significant limitations and areas of 
needed improvement. As previously mentioned, the strong emphasis currently placed on the accountability purpose of 
ELP assessments neglects the importance of other assessment purposes, particularly that of supporting English language 
teaching and learning. A nationwide English as a second language (ESL) teacher survey and focus group study indicated 
that current ELP assessments fail to provide teachers with results in time to inform placement decisions or high-level 
instructional support for ELL students (Wolf, 2011). The current ELP assessments have also been criticized for their limited 
usefulness in relation to content assessments. That is, ELP assessment results are not systematically linked to content 
assessments, and thus are not effectively utilized to understand ELL students’ academic performance (Rivera, 2014). 

Another major limitation of current mainstream ELP assessments lies in their underlying constructs. As described 
above, ELP assessments are needed for various purposes such as initial identification of ELLs, annual summative assess¬ 
ments for accountability purposes, and diagnostic assessments for instructional planning. However, current state and 
commercial ELP assessments have not been developed on the basis of a common ELP construct framework. A varied 
range of language skills and foci are manifested in states’ ELP standards, which has led to variability in the ELP constructs 
of assessments (Bailey & Huang, 2011; Wolf & Farnsworth, 2014). The lack of a common ELP assessment construct lim¬ 
its the inferences that can be made about ELL students’ ELP, and the guidance that can be provided for instructional 
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Figure 1 Overview of a proposed English language proficiency (ELP) assessment system. Adapted from “ Creating a Next-Generation 
System of K-12 English Learner (EL) Language Proficiency Assessments,” by M. C. Hauck, M. K. Wolf, & R. Mislevy, 2013, Prince¬ 
ton, NJ: Educational Testing Service. Retrieved from https://www.ets.Org/s/research/pdf/24473_K12_EL_Paper.pdf. Copyright 2013 
Educational Testing Service. 

support. The next section describes our proposal for the conceptualization of a next-generation ELP assessment system 
to overcome the major limitations of the current-generation ELP assessments. 

Overview of the Proposed English Language Proficiency Assessment System and Theory of Action 

We propose building an assessment system in which various assessment components are intertwined to facilitate the 
systematic use of assessment results toward the ultimate goal of improving ELL education. That is, next-generation ELP 
assessments should be conceptualized and designed as an assessment system, rather than as a single assessment or simply 
multiple assessments. Figure 1 displays a graphic representation of the proposed ELP assessment system. The conceptual¬ 
ization of the system entails not only the development of the assessments themselves, but also the creation of a mechanism 
to link the assessment results to effective and efficient assessment data uses for school administrators and teachers. Thus, 
data management and professional support components are essential components of the ELP assessment system. 

The Main Components of an English Language Proficiency Assessment System 

Our proposed ELP assessment system comprises four main components: (a) an overall ELP construct, (b) various mea¬ 
sures for specific assessment purposes, (c) data management, and (d) professional support. Below is a brief description of 
each component. 

Overall English Language Proficiency Construct 

The construct of ELP to be measured in the assessment system is the fundamental base that will guide assessment design, 
undergird valid inferences made about students, and inform teachers’ instruction. In defining the construct, both theo¬ 
retical and practical resources should be taken into consideration. We propose that communicative competence models 
in second language acquisition and learning, academic English language literature, and various standards (both ELP and 
academic content standards) should be reviewed to define the ELP construct. It is important that the overall ELP con¬ 
struct should consider the types of language needed to perform the tasks delineated in both academic content and ELP 
standards. This will ensure that ELP assessment results can provide accurate and useful information on students’ com¬ 
municative language use abilities in academic contexts. By defining an overall ELP construct, various assessments can 
be strategically developed, and their results can be interpreted more meaningfully, being linked to the overall construct. 
A more detailed description of our conception of the overall ELP construct is presented in the Overarching Construct 
section of this article. 
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Various Measures 

The proposed assessment system includes various measures to serve specific assessment purposes. These measures 
are intended to meet a wide range of needs to serve ELL students. Among baseline measures of ELP, we propose a 
screener/placement assessment and annual summative assessments for accountability. While this system primarily 
targets the assessment and improvement of students’ ELP, a measure of students’ first language proficiency will also 
provide valuable information for teachers to plan instruction that better serves students’ needs. We thus include a 
native-language proficiency measure as part of a set of baseline assessments. We further propose that the ELP assessment 
system encompasses classroom-based instructional assessments for ongoing diagnostic and formative purposes. Those 
instructional assessments may be used at any time during instruction to guide lesson planning. Summative assessments 
may also include not only an annual assessment but also interim or benchmark assessments in order to provide infor¬ 
mation about the extent to which students have developed their ELP after a certain period of instruction. When these 
various measures are developed based on an overall construct, the schools and teachers can choose an assessment to use 
for specific purposes and needs. 

Data Management System 

Easy access to assessment data and facilitation of the provision of meaningful information to various stakeholders, includ¬ 
ing parents and students, is a critical component of the proposed assessment system. Data management and score report¬ 
ing comprise a critical area that needs significant improvement over the current state of the art. Currently, students’ ELP 
assessment data are not usually linked to their content assessment data or other background information. Rather, ELL 
students’ ELP and content assessment scores tend to be placed in a different data base with a different set of background 
variables. We propose that the data management and score reporting system be designed to facilitate the effective use 
of assessment results by allowing for comprehensive and easy access to all relevant information related to a student’s 
performance on all assessments in the system. 

Professional Support 

To increase the fidelity of implementation of the assessment system, professional support is essential. The ELP assess¬ 
ment system should include systematic professional support for both ESL/bilingual teachers and those who teach content 
subjects to ELLs. The professional development (PD) support in the system may contain: (a) teacher guide materials to 
promote understanding of the assessment construct and data; (b) workshops for developing assessment best practices, 
including scoring and the uses of data; and (c) instructional resources aligned with the assessment results. 

Theory of Action 

The components of the proposed assessment system should be deployed in a systematic way to advance the ultimate goal 
of improving ELL education. In conceptualizing the current proposed ELP assessment system, it is also important to 
envision how the components of the system should work together to realize the ultimate goals of the system in practice. A 
theory of action provides a framework within which to record changes and outcomes expected to result from the effective 
use of an assessment system. It also offers a useful mechanism for deriving the validity arguments that will need to be 
supported with evidence after the operationalization of the system. We have adopted a theory of action drawn for another 
assessment system, Cognitively Based Assessment of , for, and as Learning ( CBAL ™). Bennett (2010) suggested that a 
theory of action for an assessment system should include the following components: (a) the components of the assessment 
system, (b) the intended effects of the system, (c) the interpretive claims made from the assessment results, and (d) the 
action mechanisms (p. 71). Adopting Bennett’s template of a theory of action, we present a theory of action for the current 
proposed assessment system depicted in figure 2. 

The underlying premise of our theory of action is that a set of quality assessments aligned with the standards (or 
well-defined construct or learning objectives) will help teachers not only to better understand students’ abilities but also 
to better understand the learning goals, leading to improved teacher instruction and student achievement. As shown in 
figure 2, a data management system and professional support are also essential components to aid teachers in utilizing 
the assessment results for the intended effects of improved ELL education. 


4 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


COMPONENTS 


Overall Construct 

> Language proficiency model, based on ' 
■S Common Core State Standards 
•S Other ELA & ELP standards 
■S Literature on academic language 
■S Second language acquisition theories 

Various Measures 

• Baseline assessment tools (e.g., / 

screeners, placement assessments, 
native language proficiency measures) 


i Instructional, classroom-based — 
assessment (e.g., diagnostic . 
assessment, formative assessment) ' 

i Summative assessments consisting of 
periodic (benchmark) & annual end-of- 
year assessments 


Data Management System (DMS) 

• System or tool to keep track of all data 
generated by the system & to create \ s 
feedback for teachers & students to 
guide teaching & learning 

Professional Support 
Intended to assist teachers in 
understanding & implementing the 
ELL-ELP screening, formative, & 
summative assessments, and to 
develop their knowledge and skill (e.g.,- 
teacher guides, websites, workshops, 
instructional resources) 


HYPOTHESIZED ACTION 
MECHANISMS 


Teachers use model to inform instruction. 


Teachers formulate & communicate learninc 
goals to students in terms of model. 


Teachers & administrations use screener 
& placement information to maximize 
efficient allocation of resources to ELLs. 
Teachers also use ELL students’ native 
language proficiency information to develop 
strategies to improve ELL students’ English 
language & content learning. 


Teachers use formative materials 
regularly to gather information about 
student learning. 


Teachers & students use inferences to 
adjust instruction. 


Teachers & students use periodic 
summative for formative follow-up. 


Teachers & administrations use aggregated 
summative information to make decisions 
about instructional programs for ELLs. 


Teachers use data from all assessments to 
track student progress against model. 


Teachers provide feedback generated by 
the DMS for students & parents. 


Administrators use data from all 
assessments to identify districts/schools 
that need attention. 


ESL teachers meet in communities of 
practice to discuss implementation of 
language proficiency model & use of data 
from the ELP assessment system. 


£ 


INTENDED INTERMEDIATE 

EFFECTS L 


For Teachers 

(ESL, bilingual, mainstream, content-area) 

• Improvement of ELL’s teachers’ 
understanding of 

S Language proficiency & development 
models for ELL students’ academic 
success 

S Relationships between assessment & 
instruction, leading to effective use of 
assessment data for instruction 
S ELL students’ difficulties & strategies to 
improve student learning 

• Better understanding of individual ELL 
students’ English (strengths & 
weaknesses) 

• Better tracking of progress of ELL 
students’ English language development 
(for instruction & accountability) 

• More meaningful, actionable information 
to adjust instruction to help student 
learning 

• Systematic use of formative assessment 
to make appropriate instructional 
adjustment 

• Improved understanding by content 
teachers of ELL students’ language skills 
and use of appropriate strategies to help 
student's content learning 

For Administrators/ Teachers 

• Increased adequacy of ELL identification 

• Information to facilitate resource 
allocation for ELL students' needs 

• Reliable and valid decisions about ELL 
students’ reclassification (exit) 

For Students & Parents 

• Improved student engagement in 
assessment and learning 

• Improved student self-learning skills 

• Ongoing feedback about students’ 

English language development for 
students & parents 


£ 


INTENDED ULTIMATE 
EFFECTS 


• Meaningful & useful information that 
facilitates ELL students’ learning in ELP 
& content areas 


• Improvement of individual ELL 
students’ ELP & content areas 


• Valid information for policy makers to 
make adequate decisions about ELL 
education 


Figure 2 Theory of action for an English language proficiency (ELP) assessment system. 


Assessment Purposes and Target Population 

In designing an assessment, the construct, that is, what is to be measured, should be defined with the specific purposes and 
intended uses of the assessment in mind. In this section, we briefly describe the overall goal of the ELP assessment system 
and the specific purposes of major assessments in the system. We also specify the target population for the assessments. 

The overall goal of the proposed assessment system is to adequately measure ELL students’ English language ability 
in school settings so that educators, parents, and students are provided with useful information to make decisions about 
instructional support. The ultimate goal of this system is to assist students working to develop the language abilities needed 
to successfully engage in school life. More specifically, the purposes of the system and various assessments are as follows. 

Overall purposes of the system 

• To measure ELL students’ ELP needed in school settings 

• To provide useful information about ELL students’ ELP to help ELL students acquire appropriate English proficiency 
for academic success 

• To fulfill the federal requirement to measure ELL students’ progress in ELP development for accountability purposes 
Purposes of the baseline assessments 

• To identify ELL students (i.e., to determine whether a student has a language barrier to learning in English-medium 
mainstream content classes) 

• To measure ELL students’ levels of ELP to help with placement of students into appropriate instructional programs 

• To measure ELL students’ native (or home) language proficiency as supplementary information to inform appro¬ 
priate service or program decisions for the students 
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Purposes of classroom-based instructional assessments 

• To provide diagnostic information about areas of strength and weakness in students’ language ability 

• To provide formative information for instructional planning and to guide student learning 

Purposes of summative assessments 

• To measure students’ English language development and track their progress over time 

• To provide information to help make ELL program exit decisions 

• To comply with legal requirements for accountability purposes 

It is worth pointing out that the purposes of various assessments are not mutually exclusive. For instance, these var¬ 
ious assessments are all intended to measure the ELP needed by students in school settings. However, the design and 
the scope of the construct to be measured in each assessment vary depending on the primary intended purpose of 
the assessment. 

As far as the target population is concerned, the assessment system is mainly designed to serve the needs of ELL students 
and their teachers (both language and content-area teachers) in K-12 schools. ELL students are defined as students who 
are still developing their English proficiency and need support to meaningfully participate in English-medium mainstream 
content classes. ELL students are a highly heterogeneous group in terms of their educational and cultural background. It 
is also important to note that ELL students’ first language and ELPs vary greatly, requiring ELP assessments to cover the 
wide range of language proficiency levels. Generally speaking, our target ELL population includes the following groups: 

• Early arrivals (including United States-born ELLs) 

• Recent arrivals/newcomers (including both students with interrupted formal education and students who have 
acquired academic literacy skills and content learning in their first language) 

• Long-term ELL students (students who have been designated as ELLs for over 6 years, often possessing fluent oral 
English proficiency but limited literacy skills) 

For each specific assessment, the target population needs to be defined accordingly. For instance, the target population 
for an ELL screener/identification assessment will include students identified as potential ELLs who are newly entered 
into a school. 


Overarching Construct 

In this section, we discuss the overarching construct for the ELP assessment system and the theoretical and research- 
based backgrounds that helped to define the conceptual and operational construct. Additionally, we present a framework 
to operationalize the construct and aid in the development of assessment items and tasks. 

We propose that the overarching construct for the proposed ELP assessment system be ELL students’ communicative 
language ability in school settings. That is, the assessments in the system should measure ELL students’ English lan¬ 
guage ability to successfully communicate in school contexts where English is the medium of instruction, using linguistic 
resources appropriately for a given task and context. Our key guiding principle to define the construct lies in students’ suc¬ 
cessful learning experiences. As students develop their language abilities, they should be able to competently use them in 
order to engage in various school tasks for social and academic purposes. The construct of communicative language ability 
puts emphasis on the effective use of one’s linguistic resources rather than on the accuracy or knowledge of those resources 
alone. In other words, it focuses on ability to use one’s linguistic resources appropriately to participate in communication 
in a given context. Language use is thus purposeful and interactive with other situational variables (e.g., interlocutors, 
topics, texts). 

Two broad purposes for communicative language use in K-12 school settings include 

• accessing academic learning in school contexts using foundational and higher order language skills; and 

• engaging with peers, teachers, and staff in school contexts that are not strictly content learning-focused, using foun¬ 
dational and higher order language skills. (For convenience, we have labeled these interactions “social,” although 
we recognize that academic learning is itself a highly social event, and that all language use occurs in some social 
context.) 
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Theoretical Background 

Our approach to defining the construct rests on both theoretical and empirical bases. As for the theoretical background, 
communicative competence models (Bachman, 1990; Canale & Swain, 1980; Hymes, 1971) along with functional linguis¬ 
tics (Halliday, 1978) have guided us conceptually in the development of a framework to organize the ELP construct. 

In both communicative competence models and functional linguistics, it is asserted that the purpose of language is 
communication. This view highlights the interactive nature of language use to construct and negotiate meaning. Language 
use involves the coordinated deployment of highly interrelated skills, rather than discrete skills, to achieve communicative 
goals. Lor instance, even while reading (which is typically considered a receptive skill) one constantly interacts with the 
text, and by extension the author, to construct meanings, functional linguistics focuses on meaning-based language use. It 
thus emphasizes the contexts and social purposes of language use, as opposed to a discrete structural approach (Schleppe- 
grell, 2004). Language forms are viewed as resources to achieve communicative functions and purposes (Savignon, 2001). 
Recently, van Lier and Walqui (2012) have further argued for a conception of language as a form of human action, stress¬ 
ing the need to consider the larger context in which communication takes place in order to support K-12 ELL students’ 
English language development. In contrast to traditional theories that define language either in terms of form (formal lin¬ 
guistics) or function (functional linguistics), this social action-based view reflects the current needs to educate students 
to develop sophisticated language use competencies implicated in carrying out rigorous academic standards intended to 
usher students toward college and career readiness (e.g., Common Core State Standards [CCSS]). 

While we acknowledge the interactive nature of language use and competence, we do not deny the value of identi¬ 
fying and organizing the separate linguistic components implicated in communicative interaction. Doing so is essential 
for the purpose of developing an assessment. It not only will offer a base upon which to define an operational construct, 
but also will inform the design of appropriate assessment tasks and the making of appropriate inferences about students’ 
language ability. Canale and Swain (1980) defined a communicative competence framework comprising three subcom¬ 
petencies: grammatical competence, sociolinguistic competence, and strategic competence. Adapting this framework, 
Bachman (1990) classified language knowledge and strategic competence as the two main components of communica¬ 
tive language ability. He further postulated different areas of language knowledge and organized them hierarchically, for 
instance, language knowledge comprises organizational and pragmatic knowledge at the upper level and knowledge of 
vocabulary, syntax, and phonology at the lower level. This type of organization and conceptualization of communicative 
language ability, with its various interacting components, is instrumental for both instruction and assessment purposes for 
ELL students. Identifying language components is also useful for delineating different levels of proficiency. Later, we will 
review how functional linguistics and language componential models inform the establishment of our ELP framework. 

The past four decades in the field of language testing have witnessed various approaches to defining the ELP construct. 
These have included skill-based, trait-based, task-based, and interactional approaches (Bachman, 2007; Chapelle, 1998). 
The interactional approach, based on the sociocognitive perspective, stresses the interaction of traits and contextual factors 
along with metacognitive strategies to execute traits (knowledge or competence) in a given context. As Chapelle (1998) 
noted, this interactional approach is useful when applied to define the construct of communicative language ability by 
specifying the relevant aspects of traits and contexts. Prom the interactionist perspective, what is important to measure is 
one’s ability to accomplish communicative purposes in given contexts. In this view, language competence is a constantly 
evolving process resulting from intertwined knowledge, performance, and contexts (Chalhoub-Deville, 2003). The inter¬ 
actional approach has been adopted for the design of various language assessments, including the TOEFL® test, a widely 
used evaluation of communicative language ability in the academic context of English-medium institutions of higher 
education. 

As reviewed above, the components of communicative language ability are relatively well established in the literature. 
In particular, foundational linguistic components, including phonological, morphological, lexical, syntactic, discourse, 
and sociolinguistic aspects, are recognized as key components of language ability. Having identified these components, 
specifying relevant contextual features requires a systematic analysis of target language use (TLU) domains. This domain 
analysis is also an essential step in the ECD approach to assessment development. In ECD, information gathered from 
the domain analysis (e.g., features of tasks, characteristics of individual differences) offers a basis upon which to model 
the domain of interest that we intend to measure (Mislevy & Yin, 2009). This concept is similar to that of the analysis of 
TLU situations and tasks to inform the design of assessment tasks and the making of inferences about test takers’ ability 
(Bachman & Palmer, 1996). Analysis of the characteristics of TLU tasks will allow the test developers to craft assessment 
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tasks that bear important TLU characteristics. This process will enhance the generalizability of inferences made about 
students’ language ability beyond the test-taking situation. In the case of ELP assessment development for K-12 ELL 
students, domain analysis involves the identification of characteristics of tasks that students would encounter in school 
contexts and examination of academic content and ELP standards that delineate expectations for student achievement. In 
a later section, we describe our analysis of TLU domains, including various standards based on the theoretical frameworks 
reviewed here. 


Literature on School Language 

Language testing experts and language researchers have criticized previous English language assessments used for K-12 
ELL students because these assessments did not adequately measure the development of the academic English language 
skills students need to be successful in school settings (Bailey & Butler, 2003; Bailey, Butler, Stevens, & Lord, 2007; Collier 
& Thomas, 1989; Garcia, McKoon, & August, 2006; Hakuta & Beatty, 2000). A number of researchers are now devoting 
greater attention to the characteristics of school language, taking a functional linguistic approach to advancing our knowl¬ 
edge of academic language functions and the specific linguistic features that accompany those functions (Bailey, Butler, 
LaLramenta, & Ong, 2004; Chamot & O’Malley, 1994; Schleppegrell, 2004). Based on analyses of standards, textbooks, 
and classroom observations, these researchers suggest that there are prevalent or specific language functions that students 
are expected to perform across and within disciplines (e.g., history, mathematics, science), further, the analysis of various 
language functions in tasks and contexts reveals that very specific types of language use and knowledge are required to 
perform those functions successfully. These empirical findings support an interactional approach to defining the construct 
of language ability, in that one’s linguistic choices are constrained by specific communicative purposes and contexts. 

Schleppegrell’s (1998, 2001, 2004) extensive research on school language offers concrete examples of how different 
lexical and grammatical choices are made to carry out functional purposes in response to the demands of tasks. Her 
research also indicates that the language features of conversational interaction and school-based texts differ across various 
dimensions. Lor example, in conversational interaction, lexical density tends to be sparse and syntactic structure simpler 
when compared to academic texts, where lexical density is higher, with expanded noun phrases and nominalization in 
complex syntactic structures. Schleppegrell also noted that some common linguistic features appear across tasks, likely 
due to the fact that all school-based tasks share similar purposes in academic contexts. Lor instance, while different genres 
of texts, including narratives, descriptions, definitions, and expository essays, contain genre-specific features, they also 
exhibit many common register features (Schleppegrell, 2001). This supports the notion that there are general academic 
language features and discipline-specific academic language features. Schleppegrell also found that the most common 
language functions that students in the primary grades are expected to enact include describing an object and narrating 
past events. These language functions demand the acquisition of temporal and spatial linguistic resources (e.g., tense, 
transitive words, discourse features to organize the temporal relationships across sentences). It is also notable that the 
level of sophistication of linguistic choices will be different across tasks and contexts. A child’s description of an object 
to his/her peer at the playground may be different from a description of the same object in a class presentational activity 
(e.g., during a show and tell). 

Bailey et al. (2004) also provided valuable findings from their analysis of the academic language features of standards 
and textbooks. Their standards analysis identified typical academic language functions, for instance, language demands 
and functions in science standards and textbooks include the functions clarify questions; inquire; make observations; 
organize and collect data; take measurements; use appropriate methods (e.g., math) and tools (e.g., computers); interpret; 
summarize and describe data; report on inquiries by writing, drawing, and graphing; communicate scientific explana¬ 
tions; describe and explain findings; identify cause and effect; and critique and consider alternative explanations. Bailey 
et al.’s research also found that there are different levels of language demands and language functions required at the 
different grade levels, for example, the most common language functions at the elementary grade level are to analyze, 
compare, describe, observe, and record scientific information. At the middle school level, students were expected to com¬ 
pare, explain, identify, and recognize, from their analysis of ELP standards, the researchers found that the most frequently 
expected functions were to ask, clarify, express, imitate, listen, negotiate, participate, request, and respond. The researchers’ 
observations of science classroom discourse indicated five predominant language functions including explain, describe, 
compare, question, and comment. 


8 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


These empirical findings on school-based discourse and texts shed light on operationalizing the ELP construct in 
academic contexts. In addition to the functional linguistic approach, Scarcella (2003) illustrated how a communicative 
competence model can be adapted to establish an academic language framework. She proposed that academic language 
involves multiple linguistic, cognitive, sociocultural, and psychological dimensions composed of integrated components. 
Following the communicative competence model, she also described discrete linguistic features (phonological, lexical, and 
grammatical components), language functions (sociolinguistic component), and stylistic register (discourse component) 
of academic English. 

The characteristic of school language is also well-delineated in the framework for English Language Proficiency Devel¬ 
opment (ELPD) Standards corresponding to the CCSS and the Next Generation Science Standards (the ELPD Framework, 
henceforth, Council of Chief State School Officers [CCSSO], 2012). The ELPD framework was created in part to provide 
guidance to states working to develop or adapt ELP standards for alignment with the CCSS, which were adopted by the 
majority of the states as their academic content standards in English language arts and mathematics. The ELPD framework 
highlights that identifying the characteristics of the disciplinary language used in classrooms is important to understand¬ 
ing the nature of school language in general. Disciplinary language in K-12 settings can be described in terms of the 
discipline-specific analytic language use tasks, associated receptive and productive language functions, and the roles of 
interlocutors and direction of communication (e.g., student to student, student to teacher, teacher to student, teacher to 
entire class). This view is useful to organize students’ language use and tasks in K- 12 school settings. 

Previous literature on language competence models and academic language characteristics in school settings enriches 
our understanding of the nature of language that ELL students acquire. In turn, it guides us to conceptualize the construct 
we intend to measure. In the subsequent section, we discuss an additional important source to consider in defining the 
ELP construct for our assessment system, that is, standards. 

Alignment With Standards 

In defining the language demands and required skills of school settings, existing K-12 academic content and ELP stan¬ 
dards are a useful resource. In particular, general and specific tasks and language functions that students are expected to 
perform can be extracted from analysis of existing standards such as the CCSS or the academic content and ELP standards 
that are in use in various states. The standards also provide an expected curricular sequence and learning progression, both 
within and across grades, which guide us in our characterization of the different levels of proficiency. However, we also rec¬ 
ognize that the existing standards must be treated with caution, given their wide variation in terms of their breadth, depth, 
and emphasized content. Therefore, second language acquisition and learning theory and research must accompany the 
analysis of educational standards in the work of defining an ELP construct and accompanying proficiency levels. 

An additional reason for basing the definition of the ELP construct in part on existing standards is to ensure that the 
assessments in our proposed system are aligned with the actual standards and curricula in practice. The integration of 
the language demands and skills contained in the standards into the assessment development process enables an inherent 
alignment between the assessment and the standards (Bailey & Wolf, 2012). Recent, ongoing efforts to understand the 
next-generation ELP standards aligned with the CCSS provide an opportunity to build a better understanding of language 
as a springboard to improved instruction and assessment for ELL students (e.g., see Understanding Language Initiative, 
http://ell.stanford.edu/; CCSSO, 2012). Efforts were also made to illustrate an example of close alignment between ELP 
and content standards and learning. 

Acknowledging that the CCSS describe a core set of common knowledge and skills across grades that come to bear on 
access to and success in college and the workplace, our analysis of the standards involves identifying essential language 
skills that ELL students require to meet expected levels of achievement in the standards. As the standards documents 
do not always explicitly articulate the relevant language demands and skills of each standard, identifying underlying and 
implicitly embedded language skills is crucial. The following two examples are from the Understanding Language Initiative 
papers (e.g., Bailey & Wolf, 2012; van Lier & Walqui, 2012). 

The first example comes from the reading standards: 

Integrate and evaluate multiple sources of information presented in different formats and media (e.g., visually, 

quantitatively) as well as in words in order to address a question or solve a problem. (CCSS, ELA Reading, Grades 

9-12) 
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Bailey and Wolf (2012) pointed out that this standard contains both explicit and implicit language demands and 
skills, including understanding a given question, comprehending the content of multiple oral and written materials, com¬ 
paring and contrasting the information in the materials, integrating the information with a similar or different theme, 
and evaluating the relevance of the information to the question. Depending on the topic of the question and materi¬ 
als, knowledge of domain-specific linguistic features (e.g., technical vocabulary, specific grammatical structures) may 
also be needed. 

The second example comes from the mathematics standards: 

Compare properties of two functions each represented in a different way (algebraically, graphically, numerically in 
tables, or by verbal descriptions). For example, given a linear function represented by a table of values and a linear 
function represented by an algebraic expression, determine which function has the greater rate of change. (CCSS, 
Mathematics, Grade 8) 

For this standard, van Lier and Walqui (2012) identified substantial language demands in mathematics. These include 
listening to descriptions, discussing functions with peers, and developing ways of expressing comparative information and 
results, both so that other students can understand them and so the teacher is satisfied that the student understands. These 
examples demonstrate both common and specific language demands and skills required to meet content-area standards. 
Moreover, it is our position that standards analyses should cover all major content areas, including math, science, and 
social studies — not solely English language arts. 

As described earlier, the recognition of the strong interconnection between language and content implied by the CCSS 
is clearly reflected in the ELPD framework (CCSSO, 2012). The ELPD framework describes the sophisticated language 
competencies that students need to reach the CCSS and provides a taxonomy to identify language tasks and language 
skills across content areas, including specific examples. 

Informed by the work of other researchers as well as our own, we attempt to operationally define our ELP construct 
in terms of the essential language skills that ELL students need to acquire for successful participation in school tasks. 
The next section illustrates the framework that organizes our analysis of various TLU standards and tasks towards the 
operational definition of our ELP construct. 


Operational Definition of the English Language Proficiency Construct 

As mentioned above, the construct that our assessment system is intended to measure is communicative language ability in 
K-12 school settings. Therefore, it is important that instruction and assessment be enriched with as much communicative 
context as possible. 

Using literature addressing the purposes of communication (Bailey & Heritage, 2008), we adopted four types 
of purposes for which K-12 students communicate in school contexts: social-interpersonal, school-navigational, 
general-academic, and discipline-specific language use purposes. 

Students can perform language functions (e.g., describe, summarize, analyze, compare, evaluate) for any of the pur¬ 
poses outlined above. The degree of sophistication and demands of language to perform specific language functions in 
different contexts are expected to differ by grade level. Still, this categorization of purposes allows us to specify language 
tasks and skills that students would be expected to perform not only in academic content learning settings but also in 
other school settings. 

Our organizing framework includes the language functions that are required in various TLU contexts and the linguistic 
resources that are required to realize them. These linguistic resources are organized into phonological, lexical, syntactic, 
discourse, and sociolinguistic dimensions. Figure 3 illustrates the framework. 

Identifying the characteristics of the TLU domain is a key step in supporting our claim that the performance of test tak¬ 
ers on test tasks relates to their expected performance in important situations in real life. Closer correspondence between 
TLU tasks and test tasks (authenticity) will allow us to increase the validity of our interpretations about a test taker’s 
language proficiency based on his or her test performance (Bachman & Palmer, 1996, 2010). The examples of TLU task 
characteristics extracted from the standards and literature in the categories reading, writing, speaking, and listening are 
included in the appendix. 
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TARGET LANGUAGE USE PURPOSES (contextual features include specific tasks, topics, interlocutors, functions, input mode, and text genres) 


SOCIAL-INTERPERSONAL 


SCHOOL-NAVIGATIONAL 


GENERIC-ACADEMIC 

(content learning across multiple 
disciplinary areas) 


DISCIPLINE-SPECIFIC 

LANGUAGE FUNCTIONS 

(in receptive and productive language, in spoken and written language) 



Beginning, intermediate, early-advanced levels 





Greeting; Making a request; 

Asking directions; Following directions; Asking clarifying questions; 

Locating specific information; 

Telling a story; Describing; Summarizing; 

Comprehending/interpreting information; 

Explaining; Analyzing; Inferring; Predicting; 

Synthesizing; Comparing/contrasting; 

Arguing; Criticizing; Evaluating 

LINGUISTIC RESOURCES 

(in the phonological, lexical, svntactic, discourse, and sociolinquistic dimensions, at both foundational and more complex levels) 

• High-frequency vocabulary 
words 

• Relatively simpler syntactic 
structures 

• Appropriate 
sociolinguistic/pragmatic 
language use 


• Mixed vocabulary from everyday 
and academic contexts 

• Relatively simpler syntactic 
structures 

• Appropriate use of registers, 
formality, etc. (sociolinguistic 
and pragmatic) 


• General academic vocabulary 
words for learning content areas 
and performing tasks for 
content learning 

• Academic syntactic structures 

• Academic organizational 
discourse 

• Descriptive, narrative, 
expository, argumentative 


• Technical vocabulary words 
(the concept of a work will be 
taught as content knowledge) 

• Academic syntactic structures 

• Academic organizational 
discourse 

• Discipline-specific genres 

TASKS 

(examples) 

■S A student asks his/her friend 
about weekend plans. 

■S A student asks his/her 
classmates to form a study 
group. 


•S A student locates specific 
information from a field-trip 
handout. 

S A student delivers information 
about a school event to other 
classmates. 


v"A student comprehends an 
author’s main argument in an 
extended, persuasive genre 
text. 

S A student explains the cause 
of an event in a story to the 
class. 


S A student makes a 
hypothesis about a specific 
experiment in a science 
class. 

S A student explains a 
mathematical solution in 
writing. 


Figure 3 Operational English language proficiency (ELP) construct framework. 


Operational Construct Definitions of Four Language Modalities 

As described earlier, we use an interactional approach to define the ELP construct. That is, student’s language use 
abilities should be measured with as much context as possible so that we can adequately make inferences about stu¬ 
dent’s communicative language ability. Although the assessment situation is limited, it is desirable to specify as much 
as possible about the context or situation in which the student is expected to perform, including participants’ roles, 
interaction types, communication purposes, and language functions, in order to assess the test taker’s communicative 
language ability. 

While acknowledging the inherently interconnected nature of language skills and their use, it is useful to identify spe¬ 
cific language skills for the purpose of assessment and instruction to help in ELL students’ English language development. 
As in the ELPD framework (CCSSO, 2012), identifying TLU tasks and language functions analytically also helps to design 
appropriate language assessment items and tasks. In operationalizing the communicative language ability construct for 
K- 12 students, we describe the construct for each modality (reading, writing, speaking, and listening) in general and 
follow with a description of the construct as it manifests itself in that modality for various purposes. We have divided 
the purposes for which students must read and comprehend written English into four categories, although we recognize 
that these categories are not mutually exclusive and can co-occur within the same setting; for instance, in the classroom 
students may engage with different texts for all four purposes. Then, we identify commonly expected language skills to 
provide useful information about the student’s language ability. 

To offer a language progression model, we organize specific language skills at the foundational, basic, and higher- 
order skill levels. Table 1 includes a general description of the three levels across four language modalities. Note that 
the description of each skill is not intended to be comprehensive; rather, it is intended to demonstrate one approach to 
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Table 1 General Description of Language Skill Levels 
Language skill level Description 

1. Foundational skills Definition: Foundational or prerequisite language knowledge and skills to perform any language tasks 

Examples: 

• English sound system, print concepts, sound-text matching (decoding) 

• Basic grammatical structures 

• Simple, frequent vocabulary 

• Word and phrase level language processing and simple-sentence processing 

• Processing fragmented information at the word, phrase, and sentence level 

2. Basic skills Definition: Basic comprehension and communication skills to perform tasks requiring literal 

understanding of the given inputs 
Examples: 

• Comprehending explicit meanings of texts and utterances 

• Performing basic language functions such as simple describing, sequencing, and story-telling 

• Processing information at the discourse level 

3. Higher order skills Definition: Higher order comprehension and communication skills to perform tasks requiring 

comprehension beyond literal understanding of the given inputs 
Examples: 

• Comprehending implicit meanings of texts and utterances 

• Performing higher order language functions such as analyzing, evaluating, persuading, making infer¬ 
ences, and integrating information 

• Processing information at the discourse level 


organizing the language skills to build a progression model as well as assessments in measuring students’ proficiency. The 
specific language skills for each modality are instantiated in each modality section below. 

Considering that ELL students have wide-ranging needs in terms of the various language use skills that they can benefit 
from developing, explicitly identifying the language skills in this spectrum will help teachers use the information from the 
assessment to adapt their instruction. Note that test takers will not necessarily employ these skills separately. Depending 
on the situation, a learner may draw upon multiple skills to perform a task. 

In general, our construct focuses on meaningful language use in context. Language use, as we are defining it, can be 
realized either in spoken or written form. For the purpose of operationalizing the construct for assessment development, 
we also offer a description of the construct for each modality below. It should be noted that this description is given as an 
example and is rather general. Depending on the grade level(s) and specific assessment purposes, the details of the tasks 
and expectations in the description will need to be modified. 

Reading (Written-Receptive) 

To measure ELL students’ ability to read and understand written English in order to succeed in English language instruc¬ 
tional environments. 

• Reading and comprehending texts for social-interpersonal purposes. Students should be able to read and comprehend 
written texts in order to establish and maintain social relationships. Text types for this purpose may include corre¬ 
spondence (e.g., e-mail, letters), invitations, and/or greeting cards. In addition, reading for personal pleasure is also 
included in this category. Generally, texts that can be read for leisure include novels, magazines, and newspapers. 

• Reading and comprehending texts for school-navigational purposes. Students should be able to read and compre¬ 
hend texts that are used to navigate the school environment and culture. Texts used for these purposes are often in 
nonlinear formats (e.g., maps, diagrams, organizational charts, brochures, advertisements, schedules, and written 
announcements). Reading subskills that are particularly required for this type of reading include comprehending 
explicit meaning, identifying key information, and understanding steps and procedures. 
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• Reading and comprehending general-academic texts for the purpose of learning academic content. Students should be 
able to read and comprehend academic texts in a range of genres (e.g., expository, biographical, persuasive, literary) 
across a range of subject areas (e.g., arts/humanities, science, social science). They need to be able to read such texts 
at difficulty levels up to and including those typical of what is used in English-medium classrooms. In reading these 
texts, students need to be able to understand main ideas and key supporting information, to make inferences based 
on what is implied but not explicitly stated, and to understand key vocabulary (either from previous knowledge 
or from context) and cohesive elements within the text (i.e., referential relationships across sentences). Depending 
on the nature of the specific text and the student’s grade level, students may also need to understand an author’s 
purpose, follow the logic and intended meaning of basic rhetorical structures, follow steps or directions, and/or 
identify and understand figurative language. Reading texts should not require any specific background knowledge, 
but do require students to read in order to learn new information, new terms, and new concepts in an academic 
context. In addition, students should be able to make connections across and within texts. 

• Reading and comprehending discipline-specific texts for the purpose of learning academic content. Students need to 
be able to read and comprehend discipline-specific texts (e.g., math, science, social science, language arts) to build 
content knowledge. They need to be able to read historical, scientific, and technical texts at difficulty levels up to 
and including those typical of what is used in English-medium classrooms. In reading these texts, students need to 
be able to understand technical vocabulary words and symbols; to understand and get information from diagrams, 
tables, graphs, figures, or charts; to understand academic organizational discourses and syntactic structures; and to 
deepen their understanding of the norms and conventions of each discipline. 

In reading, students should perform a wide range of reading tasks across various genres and topics (within a school or 
academic context) for different communication purposes. Some examples of key tasks and functions to perform include 
the following: 

Foundational Skills 

• Match sound to print for decoding a written text. 

• Read and recognize grade-appropriate words or phrases. 

• Understand simple grammatical structures to interpret a meaning of a sentence. 

Basic Comprehension Skills 

• Comprehend the literal meaning of a text. 

• Locate/identify specific or key information and details (analyze the development of central ideas). 

• Determine the main idea. 

Higher Order Skills 

• Infer implicit meaning. 

• Analyze the organizational structures. 

• Evaluate the information or arguments. 

• Integrate/connect information across multiple texts. 

• Compare/contrast information within or across texts. 

• Connect information to prior knowledge. 

Writing (Written-Productive) 

To measure ELL students’ ability to produce written English in order to succeed in English language instructional envi¬ 
ronments. 

• Writing in English for social-interpersonal purposes. In English-medium instructional environments, students must 
be able to engage in written communications for purposes of establishing and maintaining social and interpersonal 
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relationships. This includes the ability to write effective informal correspondence to peers or teachers and the abil¬ 
ity to recount events based on personal experience and observation, asking and answering questions, organizing, 
and facilitating shared activities and expressing likes and dislikes. Text types include, but are not limited to, e-mail 
messages, notes, and invitations. 

• Writing in English for school-navigational purposes. In school settings, students must be able to extract key school- 
related information from a variety of spoken or written stimuli and keep written records for their own future 
reference or to convey the information to another student, parent, or teacher. For instance, students may need 
to take notes while listening to a teacher’s explanation of a class assignment and then convey that information to 
another student or a parent. Student should be able to write short summaries of school-related information (e.g., 
field trip, announcements, procedures). 

• Writing in English for general-academic purposes. In English-medium instructional environments, students must be 
able to communicate in writing, using appropriate written language on subject-matter information in a range of 
content areas and genres. Students should be able to produce connected text on a range of topics and genres; to 
narrate a story coherently; to express personal opinions supported by relevant examples, reasons, and details; to 
describe a process in an academic context; to understand and be able to summarize, synthesize, and paraphrase 
important and relevant information from spoken and written stimuli; and to integrate information from multiple 
academic spoken and/or written stimuli. 

• Writing in English for discipline-specific purposes. In English-medium instructional environments, students must be 
able to communicate in writing, using appropriate written language on discipline-specific genres. Students need to 
be able communicate in writing about discipline-related topics to build content knowledge, to demonstrate mastery 
of the norms and conventions of each discipline, and to use precise language and domain-specific vocabulary to 
inform about or explain a topic. 

In writing, students should perform a wide range of writing tasks across various genres and topics (within a school or 
academic context) for different communication purposes. Some examples of key tasks and functions to perform include 
the following: 

Foundational Skills 

• Use basic features of print (e.g., upper and lowercase letters, spacing). 

• Use appropriate conventions (e.g., punctuations, spelling). 

• Use grade-appropriate vocabulary and phrases. 

Basic Production Skills 

• Use an effective writing process including planning, drafting, revising, editing, rewriting. 

• Paraphrase simple sentences. 

• Use appropriate register for a given purpose, audience, context (sociolinguistic/pragmatic). 

• Use a coherent organization (at the sentence, paragraph levels; between sentences, across paragraphs) and structure 
(introduction, body, conclusion). 

• Use appropriate details (e.g., relevant facts, supporting details, examples). 

• Construct informative/explanatory texts with a topic, concepts, ideas, and information. 

• Construct narrative/descriptive texts with details in sequence. 

Higher Order Skills 

• Integrate/synthesize information from multiple sources. 

• Summarize information from either spoken or written sources. 

• Draw conclusions in writing. 

• Analyze texts (or use textual evidence). 

• Draw evidence from sources and present evidence in writing. 
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• Compare and contrast ideas. 

• Identify relationships between ideas such as cause/effect. 

• Construct argumentative texts including claims, reasons, and evidence. 

• Construct research reports. 

Listening (Oral-Receptive) 

To measure ELL students’ ability to understand spoken English in order to succeed in English-medium schools. 

• Understanding spoken English for social-interpersonal purposes. Students should be able to listen to and comprehend 
spoken English to establish and maintain social relationships. Students should understand information that the 
speaker is conveying, including opinions, attitudes, intentions, requests, demands, and questions (both implicit and 
explicit). For the most part, listening text types for these purposes are conversations between and among students 
or between students and adult school staff (teachers, administrative staff, etc.) In the case of recorded telephone 
messages, a text may be monologic. 

• Understanding spoken English for school-navigational purposes. Students should be able to listen to and comprehend 
spoken English that is used to negotiate the school space, schedule, and organization. Listening text types may 
include announcements, instructions, schedules, assignments, and directions. The speakers may include teachers 
and other school staff (such as nurses, secretaries, principals). Listening subskills that are particularly required for 
this type of listening include identifying key information, understanding steps and procedures, and comprehending 
explicit meaning. 

• Understanding spoken English in order to comprehend general-academic content. Students need to be able to listen 
to and comprehend academic content in a range of genres across a range of subject areas. To fully engage in a class 
using English-medium instruction, students need to be able to listen to and understand monologic listening texts 
and extract main ideas and key supporting information, make inferences based on what is implied but not explic¬ 
itly stated, and understand key vocabulary (either from previous knowledge or from context). Beyond monologic 
teacher-delivered text, students must understand dialogs (or multilogues) in which they may or may not be a direct 
participant. (For instance, a student must understand a teacher’s question to another student, the student’s response, 
and the teacher’s affirmation or correction of the student’s response.) Depending on the specific nature of the con¬ 
tent, students may also need to understand figurative language (such as rhetorical devices, hypotheticals, analogies, 
literary devices), recognize digressions, and follow steps or directions. Text types may include orally delivered writ¬ 
ten texts (such as stories, speeches, lectures), extemporaneously-delivered or prepared lessons (teacher-centric), 
class discussions, field trips and tours, and recorded texts with or without a visual graphic (such as those from radio 
or television like the Discovery Channel). 

• Understanding spoken English to comprehend discipline-specific texts. Students need to be able to listen to and compre¬ 
hend discipline-specific texts (e.g., math, science, social science, language arts) to build content knowledge. Students 
need to be able to listen to historical, scientific, and technical texts at difficulty levels up to and including those typi¬ 
cal of what is used in English-medium classrooms. In listening to these texts, students need to be able to understand 
technical vocabulary, understand academic organizational discourses, and to deepen their understanding of the 
norms and conventions of each discipline. Text types may include orally delivered written texts (such as stories, 
speeches, lectures), extemporaneously-delivered or prepared lessons (teacher-centric), class discussions, field trips 
and tours, and recorded texts with or without a visual graphic (such as those from television like the Discovery 
Channel). 

In comprehending spoken English, students should perform a wide range of listening tasks across various topics (within 
a school or academic context) for different communication purposes. Some examples of key tasks and functions to perform 
include the following: 

Foundational Skills 

• Recognize and distinguish English phonemes. 

• Comprehend commonly used expressions and phrases. 
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• Understand very common vocabulary and function words. 

• Identify the meaning of spoken words. 

• Understand basic sentence structure and grammar. 

• Understand how intonation, stress, and pauses are used to convey meaning. 

Basic Comprehension Skills 

• Comprehend the literal meaning. 

• Follow directions. 

• Recognize specific or key information and details (analyze the development of central ideas). 

• Determine the main idea. 

Higher Order Skills 

• Infer implicit meaning. 

• Use rhetorical, organizational, or cohesive markers and devices to comprehend the meaning of a text. 

• Evaluate the information or arguments. 

• Integrate information with prior knowledge and/or other texts. 

• Compare/contrast information within or across texts. 

• Understand attitude, stance, and degree of certainty. 

• Make predictions based on understanding of text. 


Speaking (Oral-Productive) 

To measure the degree to which ELLs have the ability to use spoken English to successfully engage in academic learning 
and social interaction in school settings. ELLs use spoken English in school settings for four overlapping purposes: 

• Using spoken English for social-interpersonal purposes. Students must engage in spoken communications to establish 
and maintain social and interpersonal relationships. This includes speaking informally in conversations and discus¬ 
sions with other students and teachers (in the classroom and outside the classroom), recounting events and stories 
based on personal experiences, asking and answering questions, facilitating shared activities (taking turns, sharing 
resources, encouraging or discouraging others’ behavior) and expressing likes and dislikes. 

• Using spoken English for school-navigational purposes. Students need to use spoken English to interact with adult 
school staff and other students to negotiate the school space, schedule, and organization. This includes asking and 
answering questions about school activities, schedules, and locations; making requests; giving instructions; making 
announcements; describing procedures; clarifying information; and giving explanations. 

• Using spoken English for general-academic purposes. In English-medium instructional environments, students must 
use spoken English at an appropriate level of formality on subject-matter information in a range of content areas and 
genres. In dialogs with teachers and in dialogs or multilogues with other students and teachers, students should be 
able to use spoken English both to inquire about academic content (seek out information that was not previously pre¬ 
sented or was imperfectly understood) and to demonstrate familiarity with subject content (answer demonstration 
questions). In classroom contexts, students must be able to use more or less spontaneous spoken English to 

• ask the instructor clarifying questions; 

• ask the instructor for specific information or explanation; 

• demonstrate understanding of subject-matter content by answering informational questions in class; 

• give opinions on specific topics in response to teacher questions and support with relevant information; 

• participate in class discussions (such as discussing the significance of events in a story, real-world current 
events, causes, and effects), including using appropriate language to agree or disagree with other participants; 
and 


16 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


• demonstrate understanding of class material (reading assignments, teacher’s spoken explanations, video pre¬ 
sentations, etc.) by giving extended responses to teacher questions that require summarizing, paraphrasing, 
and synopsis. 

Students must use spoken English to deliver oral presentations (or combined oral and visual presentations) of prepared 
material. This includes the ability to speak in connected discourse on a range of topics and genres, to narrate a story 
coherently, to express and support personal opinions with examples, and to describe a process. Tasks may include giving a 
speech, describing or narrating a sequence of events, summarizing or paraphrasing from written or orally delivered texts, 
providing personal opinions supported by details, and integrating information from multiple spoken and/or written 
stimuli. 

• Using spoken English for discipline-specific purposes. In English-medium instructional environments, students 
must be able to communicate by speaking about discipline-specific genres using appropriate language. Students 
can demonstrate understanding of a written and/or spoken stimulus by using appropriate spoken language on 
discipline-specific topics. 

Some examples of key tasks and functions to perform include the following: 

Foundational Skills 

• Produce segmental features of English at the word level/produce English phonemes. 

• Produce English suprasegmentals (use stress and intonation) appropriately. 

• Use appropriate and precise vocabulary. 

• Use a variety of signal words and phrases and transitional devices. 

Basic Production Skills 

• Produce simple sentences using structure and grammar knowledge. 

• Use appropriate register for a given purpose, audience, and context. 

• Engage in simple exchange by questioning and answering. 

• Describe what s/he observes. 

• Tell a simple story. 

Higher Order Skills 

• Give and support an opinion using reasoning and evidence. 

• Synthesize information from multiple sources. 

• Summarize information from either spoken or written sources. 

• Paraphrase information from either spoken or written sources. 

• Explain complex and abstract ideas, inferences, and conclusions. 

• Evaluate arguments, information, processes, and results. 

Macrolevel Versus Microlevel Constructs 

It should be noted that the ELP construct described above is an overarching construct that underlies the various 
assessments of the system (e.g., identification, diagnostic, formative, and summative). The overall ELP construct also 
describes our conception of language ability. The coverage of the ELP construct will vary depending on the purposes 
of the assessments. One way of operationalizing the overarching construct is to incorporate the concept of macrolevel 
versus microlevel assessments (Bailey & Wolf, 2012; Black, Wilson, & Yao, 2011). Macrolevel assessments may serve 
mainly summative purposes, covering more coarsely grained levels of the construct to provide high-level summative 
information. On the other hand, microlevel assessments may be more suitable to serve formative assessment purposes, 
covering finer-grained levels of the construct to supply diagnostic information for daily instruction. For instance, 
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on an initial identification assessment, within this concept of macrolevel and microlevel constructs, oral proficiency, 
and literacy skills may be measured at a macrolevel. For a diagnostic assessment, microlevel constructs may be mea¬ 
sured to provide students with detailed information on their proficiency in different language skills. A macrolevel 
description may state that the assessment measures a student’s ability to comprehend various genres of text. At the 
microlevel, the assessment might measure specific language functions and linguistic knowledge, such as summariz¬ 
ing specific details and understanding certain syntactic structures. Our approach of distinguishing between defining 
the operational construct at the macrolevels and microlevels is also related to the premise that a single assessment 
cannot serve multiple purposes. For instance, while an annual, summative ELP assessment is intended to measure 
the states’ ELP/D standards, it will have some limitations with respect to covering the entirety of the standards at 
the microlevel. The construct of each assessment component in the system will be described in its respective design 
document. 

Language Proficiency Progression and Grade Levels 

ELL students have very diverse backgrounds in terms of formal schooling experience, whether in the United States or 
abroad. They also differ in their cultural experiences, native language proficiency, and ELP. Due to this diversity, ELL 
students’ ELP levels are heterogeneous both across and within grades. Thus, ELP assessments for K- 12 ELL students 
must encompass a wide range of proficiency. However, a major reason for measuring ELL students’ ELP is to estimate the 
degree to which these students can access information in grade-level materials and instructional settings. In this docu¬ 
ment, we describe three general proficiency levels that span all grades, considering the three language skill levels discussed 
in the previous section (i.e., foundational, basic, and higher order level language skills). This general proficiency descrip¬ 
tion may be thought of as a developmental language learning progression. It can also serve as the basis to formulate the 
claims for the purpose of the assessment development. Note that these levels may not necessarily map to the number 
of levels already used to make instructional placement decisions in schools, or the number of levels in states’ standards 
documents. For the placement assessment or summative assessment, the levels associated with assessment scores will be 
determined through a standard-setting study based on each state’s ELP standards. The proficiency level descriptions below 
aim to describe students’ expected performance holistically. This overall description is adapted from the ELPD frame¬ 
work (CCSSO, 2012). For demonstration purposes, only proficiency descriptors for the Grades 1 -2 span are presented 
in Table 2. 


Assessment Design Principles 

We have discussed our proposed ELP assessment system and elaborated on the approach to conceptualize the overarching 
construct of our assessments in the system. In this section, we suggest a set of general principles to consider in developing 
individual assessments within the system. 

Evidence-Centered Design 

The principles of ECD should guide the design of items/tasks and the creation of the different components of the assess¬ 
ment system. ECD helps explicate the construct that is being measured; provides a detailed description of the assessment 
items, tasks, and the rules for scoring them; and supports inferences made on the basis of evidence derived from the 
assessment system. We provide an overview of ECD in the next section, as it encapsulates the specific design process used 
in the development of the assessments. 

Balance of Discrete and Integrated Language Knowledge and Skills 

Considering the wide range of both English proficiency and experience in English-speaking environments that charac¬ 
terize ELL students, a balance of both discrete and integrated language knowledge and skills is important. This will ensure 
that the assessments are accessible to the broadest possible range of students and produce valid results for all target test 
takers. The degree of integration and specific balance of knowledge, skills, and abilities in a particular task will depend on 
the purpose of the assessment in question (e.g., diagnostic, formative, summative). 
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Table 2 An Example of the Proficiency Descriptors 


Reading (written-receptive) proficiency descriptors. Grades 1-2 

Level 1 (Beginning) • Can guess intelligently at the topics of written texts when they are accompanied by illustrations 


Level 2 (Intermediate) 

• Can begin to identify letters of the alphabet and their associated sounds 

• Can communicate in words, phrases, and predominantly simple-sentence structures 

• Can identify features of a narrative text such as title and author 

• Can use narrative text elements (title, images, etc.) to make predictions about content 

• Can read simple texts composed primarily of sight words 

• Can comprehend some aspects of written texts when he/she has relevant background knowl¬ 
edge and can draw from accompanying images to support comprehension; may miss some 
details 

• Can ask clarification questions that demonstrate comprehension of some aspects of the written 

text 

Level 3 (Early-Advanced) 

• Can read simple texts composed of sight words. Can read, if not completely comprehend, new 
vocabulary 

• Can understand typical content area texts with some support from illustrations and graphics 

• Can use features common to explanatory texts (headings, bolding, captions, images) to focus 
on main ideas and key details 

• Can elicit clarification or further explanation about aspects of text he/she does not understand 
or is interested in; questions demonstrate comprehension 

• Can make relevant connections between multiple related texts. 


Writing (written-productive) proficiency descriptors, Grades 1-2 

Level 1 (Beginning) • Can use drawings to demonstrate knowledge of a topic 



• May attempt to write phonetically what he/she can say if provided with examples 

Level 2 (Intermediate) 

• Can write simple statements using models, explicitly taught sight words, and common class¬ 
room words 

• Can label elements accurately if terminology has been explicitly taught (e.g., labeling shapes) 

• Can express original thoughts in writing when provided with models, basic language struc¬ 
tures, etc. 

Level 3 (Early-Advanced) 

• By building on the language of oral and written texts, can produce simple original narratives 
or summaries and simple arguments with supporting evidence which can be understood by 
teachers and peers 

• Can use discipline-specific terminology taught in class in his/her writing ( circle , square, triangle ) 


Listening (oral-receptive) proficiency descriptors, Grades 1-2 

Level 1 (Beginning) • Can listen beyond frustration point. Can begin to guess intelligently at topics 



• Can begin to guess intelligently at topic if provided guided listening instruction (e.g., illustra¬ 
tions, gestures, and/or background knowledge) 

• Can respond to questions and instructions in ways that signal emerging comprehension 

Level 2 (Intermediate) 

• Can comprehend main points supported by gestures, illustrations, and other scaffolds 

• Can continue to listen attentively even when material is not completely understood 

• Can manifest comprehension by using memorized utterances, gestures, facial expression, and 
intonation 

Level 3 (Early-Advanced) 

• Can comprehend almost all key points of oral texts with minimal support or scaffolding (though 
they will still benefit from it if it is employed) 

• Can manifest comprehension by posing questions, employing memorized utterances, gestures, 
facial expressions, and intonation 
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Table 2 Continued 


Speaking (oral-productive) proficiency descriptors. Grades 1-2 

Level 1 (Beginning) • Can use basic utterances and memorized language chunks with gestures to communicate 

• Can express agreement or disagreement with gesture, basic utterances, memorized chunks, and 
intonation 


Level 2 (Intermediate) • Can employ model sentences to communicate original meaning 

• Can request information or clarification, explain his/her observations, and express original 
meaning by imitating language models employed by others and by supporting speech with ges¬ 
tures, facial expressions, pictures, memorized language chunks, and basic language structures 


Level 3 (Early-Advanced) • Can produce questions, instructions, explanations, narratives, and simple arguments with sup¬ 

porting evidence which can be understood by teachers and peers by building on the language 
of others 

• Can use discipline-specific terminology taught in class in his/her explanation (e.g., liquid, solid, 
gas) 

• Can use, but does not rely heavily on, supports such as gestures and memorized utterances to 
communicate 


Note. The label of each level is provided as an example. 


Balance of Foundational and Higher Level Language Skills 

Items and tasks should be developed at different levels of difficulty and complexity to represent the entire range of pro¬ 
ficiency levels that we expect in our target test-taker population. Having a range of tasks at different levels of diffi¬ 
culty and complexity will help ensure that the assessments produce scores that provide valid information on whether 
students have attained a level of English proficiency that is necessary to benefit fully from English language academic 
instruction. 

Enrichment of Context and Input to Elicit Meaningful Language Use 

Since the focus of the assessment system is on students’ language use in school contexts, it is desirable to replicate TLU 
context features in the assessment situation. This will, in turn, help to elicit a more authentic sample of the knowledge, 
skills, and abilities that test takers might be expected to make use of in real-life communicative situations. The likelihood 
of making appropriate inferences about students’ communicative language ability from the given assessment will thus 
increase. 

Provision of Different Levels of Scaffolding Items 

One of the purposes of the assessment of ELL students’ language proficiency is to gauge the extent to which students 
may need additional instructional support to complete a given task. In an instructional setting, scaffolding strategies are 
commonly used and have proved effective with ELL students. The idea of providing scaffolding will be integrated into the 
proposed assessment system by designing task types with different levels of difficulty, complexity, and transparency. It also 
can be implemented in using an adaptive or multistage test design. 

Use of Technology 

Technology should be an essential component of the proposed ELP assessment system, as it allows for the creation of 
context-enriched, innovative items and tasks designed increase students’ engagement and interest in the assessment tasks 
and thereby elicit more meaningful and relevant evidence of the students’ communicative language abilities. Technology- 
enhanced features may include (a) automated scoring, (b) a platform for relatively quick score turnaround and feedback, 
and (c) applications enabling easy access to data and effective data use. 
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Overall Evidence-Centered Design Process 

ECD, developed by Mislevy, Steinberg, and Almond (2003), has been widely used to develop many current educational 
assessments by providing a conceptual design framework. Throughout the assessment development process, ECD can be 
employed as a coordinating mechanism for definition of the construct to be measured, specification of test items/tasks to 
include, and identification of what performance characteristics to score. ECD allows for systematic, evidentiary assessment 
development from defining the construct (based on domain analysis) to developing models to relate collected evidence 
to test takers’ abilities of interest. More specifically, three models are specified in ECD; the student, task, and evidence 
models (Mislevy & Yin, 2009; Mislevy et al., 2003). 

In what follows, we briefly illustrate each model in general terms and describe our approach to defining each model 
for the assessments in the proposed assessment system. 

Student Model 

In the student model, the claims to be made about students’ knowledge, skills, and abilities are specified. The purpose 
and the construct of the assessment facilitate the specification of the student model. To measure K-12 ELL students’ 
communicative language ability in school settings, our domain analysis included previous literature and various standards 
for K-12 students. The analysis helps us to better understand the multicomponential and highly interactive nature of the 
communicative language ability construct, as well as prevalent task types that students would encounter in school settings. 
While an overall claim about students’ communicative language ability in school settings undergirds the student model, 
individual claims about ELL students’ language ability at different grade levels and at different proficiency levels should 
also be delineated in student models for each assessment in the system. The three levels of proficiency descriptors described 
earlier provide an example of our student model. 

Task Model 

The items or tasks used to elicit evidence about students’ abilities are specified in the task model. Test specifications and 
blueprints are common artifacts describing the task model. In our domain analysis, TLU tasks are identified along with 
the characteristics of TLU tasks (see the appendix for the examples of TLU tasks). Some major characteristics of TLU 
tasks include communicative purposes/language functions, interlocutors’ roles (teacher, student, one-on-one, one-to- 
many presentation, etc.), and the linguistic complexity of the input and expected output. In developing the task model, 
these characteristics of TLU tasks should be taken into consideration so that appropriate inferences and claims are made 
about what students know and can do with language in real-life situations. 

Evidence Model 

This model concerns ways to collect evidence and develops an appropriate measurement model to relate the evidence 
gathered by the assessments in the system to claims made about the test takers. The evidence model entails determining 
important characteristics of student performance (i.e., scoring specific tasks) and analyzing the way performance across 
several tasks can be combined to support specific claims (i.e., various statistical models and data analyses). Commonly 
used measurement models include classical test theory, item response theory (IRT), and structural equation modeling, 
to name a few (Mislevy, 2011). In our assessment system development, various models maybe applied depending on the 
assessment design. 

As mentioned previously, the different components in our assessment system should focus on different aspects of the 
overarching construct at either a macrolevel or microlevel. There are three main components in the assessment system: 
(a) the initial/placement assessment, (b) the summative assessment, and (c) the formative assessment. Scores produced by 
the initial placement assessment will make implicit claims about whether students should be designated as ELLs and what 
type of support services might be appropriate. This constitutes an indication of the students’ productive and receptive 
ELP at a macrolevel. Classical test theory will be used to generate scores on the initial placement assessment. In addition 
to providing continuous numerical scores, it is also important to set cut scores indicating whether students should be 
designated as ELLs. Data should be collected from both ELL and non-ELL students, and statistical procedures such as 
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discriminant analyses may be used to assign weights to test items in order to differentiate these two groups of students. 
A cut score setting panel consisting of experts and practitioners in the field should be tasked with the determination of 
the cut score, taking into consideration all the information available on these two groups of students. 

For the summative assessment component, we want to allow stakeholders and policy makers to understand how 
ELL students are performing as a group, especially for accountability purposes. The summative assessment will not be 
useful if it cannot produce scores that can be used to satisfy various accountability systems and requirements. Under the 
current Elementary and Secondary Education Act (ESEA) Title III requirement, states need to show ELL performance 
in reading, writing, speaking, listening, and comprehension. States also need to set up annual measurable achievement 
objectives (AMAOs). These requirements demand a summative assessment system on a common scale, so individual 
student performance can be tracked over time. They also require that domain scores in reading, writing, speaking, 
listening, and comprehension be reported, as opposed to (or in addition to) a holistic proficiency score. In order to 
satisfy these reporting criteria, an IRT model offers the best fit. It will allow us to create a common scale across grade 
levels (spans). In order to make the summative assessment more efficient, a multistage assessment model might be 
considered as well. 

On the other hand, the main goal for the instructional assessment component (e.g., formative assessment) is to allow 
teachers to incorporate assessment into everyday curricular activities and produce immediate learning evidence on 
microlevel constructs taught in both pull-out and push-in ELL contexts. Information collected from the instructional 
assessment component should cover a wider spectrum than that of the other two components of the system. The number 
of attempts, the amount of time it takes students to complete individual tasks, and other information usually not included 
in traditional assessments may be collected and used to build a profile of student proficiency that allows teachers to make 
instructional adjustments or identify new paths. Structural equation models and other statistical models might be used 
to best represent the path for successful mastery of specific constructs. 

In summary, we recommend that assessment development should take place using a thorough ECD process including 
the following steps: 

• Define the purposes of each assessment and the construct to be measured. 

• Determine what claims will be made about students. 

• Identify specific behaviors that will provide appropriate evidence supporting these claims. 

• Design tasks and item types that we hypothesize will provide such evidence (and at the same time will model good 
instructional and learning practices). 

• Try out tasks and item types to determine whether the evidence provided by student performance on the exemplars 
of tasks is an adequate sample of the evidence specified as being necessary to support the claim. If proposed tasks 
do not provide adequate evidence, either (a) revise the scope of the claim or (b) try out different prototype tasks. 

• Design analytic and logistical models that will be used on the assessment. 


Validation Research Areas 

In order to support the adequacy and appropriateness of the intended test uses of the proposed ELP assessment system, it 
is essential to collect validity evidence from diverse sources. In formulating validity arguments, some examples of warrants 
that need to be examined include (but are not limited to) the following: 

• Scores on a given ELP assessment reflect the degree to which ELL students have attained a level and complexity of 
ELP needed to fully participate in English-medium school settings. 

• Items and tasks on the assessment reflect the ELP constructs defined in the system. 

• Items and tasks on the assessment have a range of difficulty and discrimination power appropriate for use in place¬ 
ment decisions. 

• Test scores provide information about students’ progress in ELP attainment over time. 

As a framework for gathering evidence to validate the assessment score interpretations and uses, we take an interpretive 
argument structure approach (Bachman & Palmer, 2010; Kane, Crooks, & Cohen, 1999; Mislevy et al., 2003; Toulmin, 
2003). This approach requires articulating an interpretative argument by making explicit the chain of inferences that 
link a specific assessment to its intended uses. Each of these inferences is then examined through collection of evidence 
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that can support or refute the claims. The validity framework employed by Chapelle, Enright, and Jamieson (2007) for 
the TOEFL validity arguments provides a useful guideline to utilize the interpretive argument approach. Chapelle et al.’s 
framework elaborates six inferences that must be supported through empirical evidence: domain description, evaluation, 
generalization, explanation, extrapolation, and utilization. Table 3 illustrates some types of studies that are appropriate for 
the accumulation of evidence to support the interpretive argument for the validity of the uses of the assessment system. 
The collection of validity evidence will be an ongoing effort continuing through the design, pilot, and operational stages 
of the ELP assessment system. 


Concluding Remarks 

Over the past decade, the assessment of ELL students’ ELP has received increased attention due to its importance 
for accountability purposes and ELL designation, as well as the shift in what is actually being measured. While the 
current ELP assessments are more advanced compared to the traditional pre-NCLB ELP assessments, a number 
of areas for improvement are still evident. In this article, we have presented a few major limitations of the cur¬ 
rent ELP assessment practices and discussed the need for developing an ELP assessment system, not just multiple 
assessments. 

We have proposed one way of conceptualizing an ELP assessment system in order to facilitate a closer link among stan¬ 
dards, assessments, and instruction for the ultimate goal of helping EL students improve their ELP and access to content 
learning for academic success. We elaborated on the overarching construct that underlies the proposed assessment system, 
as well as on the major design principles that should be considered in developing the system. Particularly, we emphasized 
the importance of articulating the construct and a framework to operationalize the construct for the assessment system. 
We hope that the suggested framework for conceptualizing the system, construct, and validation research areas will prove 
to be useful resources for those who intend to develop an improved assessment system and promote discussion of how to 
better use ELP assessments for the benefit of ELL students. 
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Target Language Use Characteristics 

The source of the standards is noted in the parenthesis next to each standard, including the state or Common Core 
State Standards (CCSS) and grade level. The standards reviewed to identify the sample target language use (TLU) tasks 
include the CCSS, California (CA), Florida (FL), New Jersey (NJ), Texas (TX), and World-Class Instructional Design and 
Assessment (WIDA). 
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comparison/contrast, cause/effect, 2. Identify some significant structural # Discourse: a coherent text 

chronological order, argument/support, (organizational) patterns in text, such # Register: formal 

lists) and text features (main headings as sequential or chronological order 

with subheadings) and explain their an d cause and effect. (CA, G3-5) 

impact on meaning in text. (PL, G8) _ 






Standards 

Task ELA ELP Language functions Linguistic forms of the input 


M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


32 


a ■ 

3. 

o 


oo 

w-a 

< § 
U ■£ 

v —' o3 
• N 


s ■ 

bo 


CL, —• 

if 

d <U U 
d 
r 

bo ^ i> 

<u * a 


■§ ® 
03 d 
-r-t 2 


o S 

s § 

"d ST 

O 

C/3 1/3 

’£* ? 

03 03 - 

d* y 
d w> 
o ° 
u o 

* s 

<u o 
d b 


O o 


’5b d 
o <l> 


o 

'H* ’ 


3 « 
« 2 


-zf to 

X ^ 
oJ O 
d .2 

U o3 
U 

o £f 

l* X 
Ci* pp 


O £ 
Cl, m 
£ 2 
£•5 

C/3 ^ 

' ^ 
G -2 

3 Qj 
? > 
d 

.2 ^ 

"S' 

o .2 
Cl* d 
o Jp 
o ° 

Cl* 1/3 

bb'S 
<Li § 


<Z) 




pp 

& 


o3 

Cl* 


3 

2 

o 

PP 

d* 


o3 


> *-< 
pp u 


hP 

Ph 


bD 

a 


& -I 
+1* 03 
2 a 


*fc! 


<u 

3 

o3 

d 

d 

bD ^ 
03 CO 

^ 1 
2 ^2 
£ o 


d 

d 


PP CL, 

^ 2 
O 


d 

fl 


<u <u 

« & 

£ .§ 
O 1/5 


PP 

O 


a> 


a §> 2 
B 2 g 
■'h g, "S 


o3 

PP 


2 b» 

2 d 

2 S 
2 "2 


-2 § 
t>b bo 
d d 
a> 03 
l-P (-1 


bo 

.d 

’2 

a" 


£ O 


Q £ 


5P >- 
CL, -C3 

2 P2 
O d 

° -3 
^ bG 

2 *3 

<u 


d 

d 

o3 

d 

o 


d 

d 

o3 


d 

d 


pp 

<u 


d 

d 

03 

CL, 

X 

<U 

d 

d 

03 


o 

•*—I 
"2 .2 
•d a 

Sr 2 

"2 .tn 
d *H 
2 £ 

2P be 
.2 .2 
o « 
"S, -3 

a > 

c <u 


03 

d 


^ 'd 

QJ <U 


'd 

®p 3 a 

.a -a a 

a a o 
S 3 LE 
^ d 


o3 


•» "d 
bo d 
.2 ^ 
’2 "2 
o 2 


-Q 

O 



C/3 

OJ 

d 

M3 

d 

<u 

’a> 

<D 

d 

d 

G 

(U 

V* 

-d 

03 

L* 

o3 

2 

-d 

.C/3 

d 

bo 

2 

Tj 

■— 

<U 

d 

o3 

-2 

d 

o3 

CD 

_d 

o 

v2 

<u 

-d 

o 

d 

<u 

o3 

d 


o 

(13 

a> 

d 


■ZJ M/ 

S Q 


& "5 


'd 

<u 

d 

03 ^ 

> 00 
-2 O 

aj 

h co 
.i=l oo 

d <J 
2 U 
-d ' 

^ , d 
<u <u 

N U 

d ^ 

§p! 

£ 


d 

d 

o3 

o3 

d 

d 


d 

d 


ci< 

d 


'S 

‘3 

bO , 

d 


d 

d 

03 


o S 

hP o 

&I- 

'd 

d .2 


oo ^ 
O ^ 


^ d 

—^ 03 

•—j 

d oo 


OJ 


o d 
■ .2" 
d *S 
o -2 


<u 


bo 1 
_d 

d 

Cd 

o3 


<D OJ 
Lh > 

ci*d 


Ci* ^ 
ci* d 
ee3 

§ a 


H 


3 -Q 


G "5 .» 

•d 2 ^ 

o ° O 

& S' rS 

O <u ^ 

dd 5 

cO 


(U 

Lh Cl* 

O QJ 
&H 3 

§\cp 

2 o 


§ E 
1 I 


g « 


PP 


>- 


d 

o 


d 

bA 

d 

03 


'bb 

d 

pp 


bo 

o3 

d 

bb 

d 

o3 


be 

d 

PP 


< 

kP 

pp 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 






Target Language Use Characteristics: Writing (Written, Productive) 


M. K. Wolf eta/. 


Building a Framework for a Next-Generation ELP Assessment System 


c 

a 

o 

Q 


-£3 


o 

Pd 


a rS 


G 

X! £ 


S3 

-9 


a tj 
& * 


Jh aJ 
•£ S3 
bO bO 

a a 

<U 03 

H-l hP 


-a 

o 


H-i O a. 


Q cd 


.§ a 


js o a 

Cl, Cl, c/3 
X X « 

w w Q 


o 

G 

S3 


S3 

a" 

G 


bo 

G 


Pd 

'a ^ 
a £ 


OS 

bh -G 


o Pd 

>- J>- 
03 T3 

a S 

> <£ 
03 


03 

-g 

'Td 
G 

o3 

<U § 

1/3 qd 

o T3 

& 

=3 1* 

CL, O 

tj ^ 

& a 

oS O 

'5 
t: 9 


jg 

T> 

G 


£ o 

g Q^ 

H <L) 
n tq -o 

<u G: 

C/5 

o 

&H « - 

H bo c« 
S3 c3 & 
Cl, -S3 _9 
T* Td Td 
O " H 
^ <D 
5P xi b 

pd u G 
l 3 <L> o 


^ G 
G O 


Td 

bb 


<D 


o 5 


<u <P 
nd £ 


<U 00 


►r y 

£ Lh 




■M <U 
03 -H 

■S 

Td 9 
G 

o3 ^ 
+_, <U 
03 Td 

a -a 

£ o 
cP £ 


K < 

,-P 
PJ 

H-l 

PP 


Td 

G 

O 


!> *“• 
> <U 

03 <u 

^ 13 


03 Td 


<u aj b 


<u 


Cl, o 
Pd M 

<U _. Sh 
*-< T3 o 

}h <u qd 
0,0 0 
o rg .5 
<u u n 


2 T3 
TO <u 


O G 
& Td 

a G 


3 S 

3 g 

^ OJ 

x s 

b § 

s -' o 
o 

bO^d 

G <u 


G 


-G 

& 


^ O 

G -G 


> G 

(U <u 


^ "S ^ 

r, r jj 


G 

-9 


Td 

o3 


G tj 

X s 


_G o3 
•£ G 
bo bO 
G G 

<U o3 

H-l l-P 


■S 


G w> g 

G .S -G 


cX fS 


s u 

e &, 

Oh 03 

O -o 

QJ O 


oo <S 
vi C ■ 
O -g 

b s.' 

>w d 

C T3 

b s ■ 

^ 5b : 

. bg 
'l-h 


o 


ir cr r 
-> rj O 

fli H 


M-, 

o P 

Gh 

is! o 

<D Ij 
—' > 
jh a> 

§ ^ 


a 

< 

Hd 

W 

d 


> > 




bo 

t G 

^d 

G 


O 

< 

H-l 

PJ 

< 

u 


'a, 
cp -y 

^ ^ a 

qj ^3 b 

.^h H 03 

Cl, .2^ G 

<U Jh <U 

.Id <u .id 
^ is! ^ 
^ ^ ^ 


bO 

G 


Td 

G 

o3 


Tj 

G 


a 2 

kj 

<u <u 

C/3 (J 

o3 G 

<D S 
t3 ^G 

Vh - G 
5-, G 
03 bC 


o> G 
« O 

Lh £ 

oj ^ 
G Pd 

c« T3 

sG G 


03 


Sr s 

^ -P 

O Ch 


o3 

kH 

Oh 
-G ^ 

s ^ 

-2 T* 


y C^ 

a 3 

I s 

03 C/D 

P o S 


s s 


s t; 

&o o 
'BbP3 

cu 1/5 

cd ^ 




<u 


ri 6 d 

r-j y 

rS G 
bo bo 
G G 


'S g 
<u r y 

H-l O 


Q 


4) 


o3 

hP 


• • • • 


Q Gh 


Td 

G 

03 


^ dd 


o ^ 

cS ^ 
sd 

G 5 

G ’G 

s > 

a « 

G t3 
bo cu 


'G 


■M <U 

S ^ 
Cl, -G 

CP -G 
G ? 
c« ^ 
_. o3 
£ oj 

u 3 
S 


bo 


o 

CP 

o 


G -G 


qq 

G 

CP 


CP 

o 

> 


O OO 

g- o 
3 <3 

-3 o-l 
^ « 
(U H-l 

Pd pp 


CP 

o 


H-l 

Ph 


Td 

G 

o3 

bO 

CP 

O 


^G 


o3 

Pd 


^ o 

cu 

> CP" 

Q w 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


33 





M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


£ 

50 

G 

cd 


-G 

CU 

cd 


3 

■9 


G 

rG 


^ g 

cu <u 


'G 

a 

cd 


Cd 


« & 

■5 3 

50 50 

g c 

<U cd 
1-1 h4 


-G 

o 


£ O Q 


bG 


2 C7< w> £ <g 

d jf in i» a) 

w £ O Q Q 


Cu 

X 

w 




< 

u 


Cl. 

bD 

c 

o 

d. 


bC 

a 


2 '2 
■ 5S -g 
O 


*G 

O 


Ft T? 


2 -£> 
cd o 
SI +3 

a 'G 

cd <L> 
bo 9 

o 5 

S I 


<u 


B* 


'G 

a 


o 

£ 


£ > 
$ t> 

>- cd 


> 

'G 

G 

cd oo 
C* O 

in 

S < 

« _i 

43 w 


-a 

CU 

'S 

O . 

£ 

J>^ 

CU 

£ 


cu 




<u 


< 

t-J 

W 

H-I 

Ph 


-Q 

O 


bD 'Tj 

O § oo 

^ «r o 


tu 

O i-l 
Cu P-1 


G 

JG 


-G 

CU 

cd 


O 

-C 


O 


-G 

G 


O 

£* ^ 

's-h G 
cd oh 
> c2 


50 G 


O cj 

Cu G 

M H 


£ 

G 

cu 

G 


G 

'G 


u Sts s s. EfS 


g 

r, rJ <u 


-a U 
« & 
■5 3 

OR OR 

3 3 

<U cd 

1-1 G 


G 

-9 


•fl 


O 


g O Q g< 


50 G 
G O 


50 

G 


d u G 

G-< c/3 Q. 

Sf « * 

oo Q W 


o 

-G 


.cd 
oh 
G 
G 

.. O 

3 « 

w § 

ft d 

i!r> cu 

_S-H 

G t 3 
O cu 


s I 
& £ 


'G 'G 
< G 
.G 

u 
O 
'G 
G 


8" S 

13 

43 w 

^ Ph 
cd • 


o 

cu" 

G 

w 


*G 

G 


SP S 


« 3 

■2 s 

cd ’G 
9 G 
O cd 


b£> Pu 

a 


£ s 


c/5 




34 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 





M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


| 

'o 

o 


«3 

a 

o 

Q 


50 

P 

fl 

t-4 


H 


Cl. 

I s 


>- P 


P <U 
-P «» 

n a 


ftp 

Ph 


P o3 

.2 a 


C3 


-p 

Ph 

p • 


o3 


Jh ?! W> 

^ rS § 

P 60 150 

Sec 

O U o3 
ft hI hJ 


P 
.p -d 


is | 
.a a 
3 2 


-3 § 

P CO 
-P -d 


ss st -a 


a « 


j O D d 


• • • • 


“ IS 2 

a <£> -g 

^ < a> 

,Jh .. 60 

t: -g § 

g 60 130 
S P P 
O CL> o3 
ft Hi h) 


. . (H Sh Sh 

-p a p « 

U P O to 

■fl § 8 'Sb 

05 d ‘-C ^ 

j O Q ^ 


• • • • 


£ 

nd 


.a "a 

a S 

<U Lh 

s -a 

S 1 


l N 


a 


-d ^ X 


. ^ 
Ph P 
<u 03 

& 99 


p a 
o .2 


^ 60 w d, 
'-£ Vi <L> P* 

a o q w 


< 

o 


00 

s 


ob 

i So -2, 

I ^ s 
■ °S o 

! ^ p 
LLl 

i ^ CJ 

! & g 

i <N 


S S 


P 


M 

o 

Cl 


o3 Jd 
Ph U 

* & 
■5 p 

bO DO 

P P 

CL) o3 
hP _1 


is | 

■a | 

j o 


Sh Lh 

d <u 
o tS 


• • • • 


v3 ^ 
■p p 


< 

PI 

X 

H 


Ph 

.p' 

-<d 


cS i-P 

O CJ 
' £ 


< 

h4 

w 

X 

H 


O «* 


Ph 

&:■ 


c « 
■g.T3 

a § 

Sh 4> 

o g 

bO Ph 

p p 


^ o _a =i 


£ 

O 

-P 


o s <d a a 


-p a 


p a 


o p 


■5 £ 


bO^ 

OtI 


p - 
<L> bO 

p .3 

<l> c 

"p -§ 
p 


^ P al 


Ph 

a 


« 

a 


s Jj y 

■S a t3 

Ph 2 03 


■a | 

S Li o -^ § 

T3 <i » C ,S - u 


"•S3 

g 60 60 

c P P 
O <U o3 
ft hJ J 


h- 1 O Q Pi 


• • • • 


.2 .P 
IS n 


gP P $ cr 

O tfe-Q « 


>- 

lH 

o 


73 

P 

o 


c 

° <U P 

^ p § 

.2 "p a 


a ^ 


-p 

’p 


» * So 

• ^P 

w ° -a 2 
bb ^ 5^"p 
P -S 5^ u 


P o 
■ -p 
p 


p -d 


2 S5 

Ph .2 

a- ^ 

03 ^ 

o 2 * 
bo -P 

p 2 • 

•rj Ph ' 


-d lh 
« £■' 
a -I 


o 
s < 

&w 


. . s a 
z 3 a 


al b0P bG 


> V 


o3 

^ -P 
'P Ph 
t3 

w 


a a 


p > 

3 3 fti T5 P <U 

js Ph a pq a pj 


- 'H 

Ef> O 


.5 ci o 


I fS 


Ph "P 
X d 




U 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


35 





Domain 3: General Academic 


M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


fP 

Ph 


3 

a 


^ a 


p 

-O 

TO 


CM <U 
- 2 U 

c SP 2 


<u r 3 


'p 

a 

TO 


TO 

fO 


TO 


TO fP 
CM “ 


'I 2 


a 

3 

bD 


-P 

O 


l> 

TO 

Cu 

mP 

bo 

TO 

CD 

i-l 

S-H 

o 

5 

CD 

Ph 

l> 

TO 

g 

bb 

P 

bO 





g 

5-h 

p 

P 

• 

• 

• 

• 

5-h 


1-1 1-1 


-a 

Ph 


^ fP ^ 

<u 2 o 
a to 

2 2 S 

P to -P 
'C Ci, ^ 

* « & 

-S 3 
00 00 
„ 3 3 

O <M TO 
£ ^ ^ 


^ a 


p 

-P 

TO 


TOd 

P 

TO 


TO 

Xi 


-P 

o 


to h5 '5b 

51 *2 'Z 1 w 

_i O Q 


p 

TO 

"Eh 

x 

w 


oo 

P 


2 fi « 

^ § 3 

a 5-i 

" 5 s 

O <3 CO 


o £ 

00 c 

£ b 

^ oo 


3 

oo 2 

.9 a 

£ J 


Cm •£ 


^ q 

*CM W5 <P 

* <d <d 

w Q Q 


TO ctf 


id oo 
U O 
2 c/5* 

o 00 

^ U 

£ (J 

o ^ 


J>^ 

’ob 

.P 

‘3 

.p 

§ 

o 

u 

'P 

P 

TO 

^ •! 
5—i 
TO 


Cm 

o 

5-h 

Cm 

& 

P 

<d 

-P 

£ 


P 

gG 

cd « 


I -SP 


I 

P 

F^ 

'p 

p 


p <d 
TO <D 

i t 

a" 

§ .a 

x E 

<U TO 

8 p 
p a 

<d ,<d 


'Td 

p 


tp 


.TO 


bO 

P 




C/5 


<D 


'p 

a § 

“ 3 
<d O 


o *S 

M 

o -g 

0 TO 


P P 
0) W 
9 3 

3 S 

S> <“ 

Sh +-* 

rt 3 
S > 
£ .22 
£ 9 


o £P 
co .3 
& S 




o 

co S? 

Ji O 

1 3 

2 w 

Cm 

§“ z 


Cm ^ 
<u .52 


-m oo 

S .s 

9 § 

op S 

T3 TO 

P 2 


m2 ^ 

M-h £ 
c/5 TO 

^ P 
8 3 

« M"! 

<U .3 
S O 

.3 CO 
rt rt 

3 3 

S 3 
° S 3 

~ CU 


oo 

o 

«C e „ 

i w o £ 

M "3 (G 
00 (U u 

. tj 3 

■i (U 

J ■§ &l 

.9 « s 
o-S « 

° ta 

^ 3 ‘C 

a 3 a- 

I 

a b & 

Mh CO TO 


P 

o 

CM 

«5 <L> 

<u 5 h 

■S .9 

»H 

<u ^ 

S 

-rd 'Td 

* S I 


'p 

P 


<u 


<u 


pj ^3 
^ cu 

e-g 

'“T ^ 

(fl C/5 

.g J 

to a 

a 

<3 3 

"to ^2 

o -S 

P O 
O .-P 
\£ "rd 

TO u 


00 TO 
VO P 

u 2 

Jt TO 
< X 

Q w 


p 

fP 8 

bfi +j 

P P 


« P TO 


^ & 


OJ 


<u 


3 p 

p^ 
O 5_ 
M) CM 

LO 


I- 

»> <D 
Si .>. 

° I 

<U H 
CM u 
TO O 
CM^ 

C/5 P 

S ’ 

P 
o 


CD 


P 

TO 

EP~ 

o oo 
3 O 
.2 « 
t> ^ 

<D U 

'w u 


o 

Cm 

x 

W 


< 

Q 


-p 

Cm 


bD 

P 


Cm 

x 


o 

& 


3d 

P 


a c 

o 

Cm ’-P 
X P 

« o 
P 

.2 2 
•P F^J 

y P 

CD 


bG 

p 

9 1 
3 TO 

u 3 
P O 

5-h CD 

to TOd 
P 

TO oJ 

P 

o >- 

•TO T3 
TO O 
_N fC3 

P tA 


TOd 

bD 

P 


O 

Cm 

Cm 

P 


> 

9 


2 "P 




F^ Cm jy 

g O ^ 


(D 'P 


o 

Cm 

X 


-rd 
- P 


TO | 

u o 


u 3 

5 O 
a, -3 

r 3 
oo o 

di -a 

C/i 03 

6 S 

% o 


'p 

bO 

_P 

5h 

O 

CM 

CM 

P 


^ O 
°9 to 

O .3 

„ TO 

< 'H, 

h-I TO 

PM -p 


Cm 

'P 

P 


c« 

r-v O 

oo Cm 
0 X 

W <D 
Ch <D 
i-l P 


^ 52 

5-h P To 
TO C>, C/5 


TO TO ► 

cm cm: 


</) 


^ p 

c/h CD 

2> p 

to GO 

in 


■P C/5 

P « 
« » 

s & 

-2 3 

3 s, 

CD cj 
TO 

.2 P 
§ & 
9 C 
9 S 

O C/5 


'P 

P 

TO 


CD 


CM 

O 


(D 
5-h 

Cm a 
x -5 
<0 3 

O <u 
T3 
& TO 
C J3 
TO TO 


TO 

5-h c+h 

O G 

■g 

O TO 

s a 

CD CD 

.tn P 


36 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 





M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


P 

# P 

P 

O 

U 


.p 

a 

o 

Q 


43 

% 


o 


43 

Ph 

c3 


o3 

Ph 


£ 


p 

42 

oJ 


*S 

Ph <u 

S-H y 
60 2 
o3 b 
H ^ 
o3 43 
Ph P 

« & 

■3 3 

60 60 

s 5 

a> a3 
1-1 hJ 


p 

-9 


oJ 

42 


* 


o 

43 


O Q ^ 


P u 

<U 

^ 2 

S 43 

<4 U 

a3 


'P 

a 

o3 


o3 

42 


.. 60 

•5 P 
60 60 
p 2 

<U Oj 
-1 1-4 


a 

p 2 
o •£ 
60 '+3 «S 

a cl, .y 


<u 


C3 


<u a> sh 
Ph Q O 


2 1 

<L> <u 

O 'T-J 

P 

fl 3 

^ Hh 00 

S '6 w 

T3 6 
CJ Ti W 

P H U 
'60 P U 

a ^ vs 
' g 
„ p 

a> <u 


60 

P 


60 
P ' 


§ is s 

CO CO Ph O 


c5 

43 


JP 

3h 

P 

o 


o 

Ph 


o 'p c3 


T3 

2 * 
O 

•P P 
cS 

14 


fa P 


Pi P 

-9 -S 


5 Sh 

O -1 

— <D 


~P P 
0) 

'■p 


£ o 

r/j ,P 

<u *43 

> <u 

’S W> 

2 .a 

5-1 53 

I 2 


P 

> 

(U <u 

3 'P 

<u 


O 

-P 


-P 

Ph 

oj 

H 

60 

O 

3 

o 


Ph 


o 

3' 


<D 


-P 

& 

in 

60 

O 

3 


O * 

~ <u 

< 3 
w s 

u 1 


I 


VO 

o 

Q 


3 a 

60 rt 

P 2 


Co 


Ph 

T3 

P 


> "P 
5 p 


60 

P 


P _, 

<U X3 

> P 

<U 03 


., o 

Ph <U 


—hh " 

Ph43 
03 CL, 


o u 

Ph 

a - a 

60 


43 

O 


3 g 

<U r H 

H-l O 


Q p4 




SfS 


(3 P cd Ph Ph 


§ o 

& < 
P t-4 
co W 


43 

Ph 


44 

O 

Ph 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


37 





Task Standards Language functions Linguistic forms 

Write analysis of 1. Draw evidence from literary or informational texts to support analysis, Analyzing Form: a written essay 


M. K. Wolf et al. 


Building a Framework for a Next-Generation ELP Assessment System 


a 


d 

■s 


'd 

d 


43 

Oh 

TO 


d 

o 

u 

'Td 

d 

03 


TO 

42 


43 

O 


“ bO 

to ^ 
bO bO 

d 2 
a> to 
i-4 




a 

o 

u 

top "a 
S o 


* d £ 


■a £ . ^ 

U c^U W 


TO 

.a S 


a 


'd 

a 

03 

S3 

O 


a 

43 

1 


oo oo 
O ' 

. VO 

< O 

w <C 
h Q 

Pc t? 


PC 

1-1 

w 

C 

o 


b 


c s > 
pc H .2 


cc 

03 

& 


™ TO 

co d 

<u cu 

a 

43 


i % 

<u ^ 
oj o 

CU x-v 

bo cu 
to b 

I! 

cc3 2 


bo 

.d 

'n 


CU -P TO t3 


£ B 

cu <_P 
bo33 

o3 .5 

bO CO 

C3 cc3 


o3 


cu 
u 

j£- § 
S S 
<5 U 

(N 


nc 

S JJ 
S a 
S & 

o o 

O U 


a 

Jh CU 
03 CO 

"to * 
<3 cu 


'd 

a 


43 

£< •• 
TO CO 


K. V' 3 

>3 

$ 2 


S3 

O 

o 

'd 

03 


o3 

42 


43 

O 


'I 2 


o 

Hh 


a 

O3 cj 
bo co 
Jh <U 

O Q 



o 


O 


"S 

S a 

b0 

C 

■S .2 

rt ^ 

<u 

PP ° 

S -3 

OJ 

o s 

cd 

U PP 


Jh 03 

Cu =3 


bo 

S3 


03 

Cu 


43 
bo 1 


.13 *d 

& a 


> ^ Cu 


nd 

T3 

a 


'd 

a 


bo 

o3 


S3 <43 
o « 

Cl, *3 
ST # 


34 

h4 

W 

< 

u 


43 

9- 


42 

O 


43 

& 


4^ 

CU 

Cu 

< 


CU CO 

^ P 

Cu cu 

2 a 

bo <u 
.2 ^ 


bO 

d 


o 

£ 

TO 


Jh 

I 


oo n 3 

vi § 
< 


C4 <5 




> s 


42 <u o3 ^ 

«? Z c "2 

.s s ; S 


2 "P> 
S3 <u 

2 p 

’■d <42 
03 cu 


N 


O 

43 


o u 

< 1 

w -2 

I-s U 

Z « 

13 

2 -2 

.2 a 


<j « 

3 -S 


o T3 


0 

< 
. H 
w 


O T3 

V .3 
S u 
b-5 

h 3 


bo 

cu 

TO 


TO P 


bo co 
d “ 


^ o 

^ C4 


CU 

uA 

Lh 

a 

5 

cu 

cd 

> 54 

^ Jc 

bo 

TO 

TO 

CU 

Lh 

a 

5 

CU 

?• 

OS 

• 

• 

• 

• 

a |p 
is a 

bO 

d 

• 

• 

• 

• 

a 

Lh 


43 

S' 


l-i d 
03 CU 


d 

o 

U 

'd 

d 

o3 


O 


o3 

42 


43 

O 


33 d 
bD bO 

G B 

<U o3 


OJJ W r -3 

2 k_] O Q ct^ 




o?5 

Lh 

_o 


d 


d 

o 

d 

bo 

d 

:g 

_o 

TO 

d 

-2 

bo 

_d 

-S' 

TO 

_N 

'o 

-a 

CU 

<u 

Jh 

Cu 

cu 

CO 

<U 

-d 

o 

'H, 

X 

"to 

d 


cd 


W 

<< 

CO 


CTO 


CU 'Td 

'B P 

CCS 03 
Sh 

bo jh 

^ 43 
Td -H 

a ^ 

M Lh 

1 w § 
b a 


bo 

d 


be 

_d 

'd 

d 


be 

d 


> OS 

cu ^d 


cu£ 

O 


d p 

a ^ 

'd 'd 

- <u 

? 3 

o o 


< 

Q 


d 

a 

bo 

d 


o 

S=L 


> 42 


o3 CJ cS 


I .13 

VO 

O ^ 


d 

o _ 

■d lh bJD 

c3 top 'Td 

> O u 

« 3 ^ 

•13 P b 


d 

43 


bo ^ 

11 

T-3 ^ 


'd d 

1 P -2 

03 T3 

I/) p. 


00 cu 

O 3 
cc i 

i—51 

pq o 
K-] 'ts 

P4 


o3 


l-H ►> CU 4d P 

U ^ t3 d P d 


a> 


d 
o 
oo 
33 

"a3 T3 

co o3 


o3 


a> 


^-3 

^n S Cb 
cu d U 

CN 


I 

VO 


o 

C4 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


38 





M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


G 

t d 

g 

o 

U 

.G 

cO 

a 

o 

Q 


cO 

-G 


-S' 

& 

L-, 

bO 

3 

3 


a 

CO 

3 

-Q 

to 

.g 


<u w> 
u a 

.a ^ 

cl 3 
d T> 

2 a 


O 


CL 

o 

i—1 <u 

« 3 

Z cP . 


<L) 


(U 


CL, "O 

jr g 


cO 

.2 S3 

bX) c/5 

O <u 


> 

'5b 

T3 

g ^ 

cO 00 
yT I 
cO VO 

-s 

CO {T3 

G W 
'5b < 
'S u 


CL, Lh 

2 a 

CL, O 

"3 

Co <u 

3 I 
? a 


^ PC 


z ° 

CL, 

^ 2 

d 3 

cd cO 

a .2 


cO 


O <u 

-C 


2 ,3 
§- a 

i i 

.2 si 


P bO 
_d 

'S 3 
o 3 

d O 
cl, d 
a, o 

CO u 


-G 

Cl 

cO 


cO 

CL 


5-1 d 

cO <U 
3 <* 

-Q X 


G 

O 

o 

T3 

d 

cO 


X M 


d (J 
<U cO 
^ 2 
> cO 

f. M 

<£ <J 
« & 

£ 3 

00 00 

c 5 

<U CO 
hd I-) 


cO 


-G 

o 


a3 2 '5b 

CJ 'jr 1 «u 

O Q rt 


a a 

.2 ° 

I I 


o?S 

G 

O 


Cfi aJ o 

N <z> 
G .—, n> 


S-i 

8 & a 

Q « < 


a p5 


-Q 

o 


S? 3 
Cl ^ 

co Q 


G 

T3 


Cl G 
r O 


O °0 
^ I 
G VO 

•3 O 
« <? 
« Q 

3 t? 
.2 ;> 
to 3 


G 

_o 

to 

§• 


O Q 


S 3 
■a ao 

a .a 
a ji 
a g 
3 a 


CO 


I 

-G 

_d ^ 

CO "cO 

3-i 2 

x 

w 


<U <U 

-P 3 
o 

Sh 

Cl 


bo 

cO 

G 

W> 

G 

co 


bo 

G 

W 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


39 





M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 




ba 

’2 


cd 

e3 

£ 


ba 

ed 

P 

bA 

d 

ed 

H-l 


e2 


# d 

a 

o 

Q 


bo 

ed 

d 

bA 

d 

cd 

t-1 


_cd 

■s 


cd 

43 


cu 

I ,—I 

<u cd 

■S 6 

U O 
cd 

.. d 


cd 

43 


oo 
43 § 

bx> 
d d 
<u cd 


Q 25 


^ d 


o 

o 

43 


•rt cd 
cd .N 

cd 

a 

a 

d 


o 

d 

o 

'■d 


bX> 

d 


d 

d 1 

d 


bX) ^ 
7 ; 00 

^ I 

C/5 

Dh O 
d 

2 < 

& Q 


'd 

d 

cd 


< 

u 


o 

d 

bO 

d 


d oo 


d 

o 

CL, 


3 1 p 

h d 


■2 3 

Jp* 'd 
> § 
d o 

CU d 


£ 

c/5 

Dh 

CU 

cj 

d 

o 

o 

'd 

d 

cd 


cd 

X> 

Lh 

1 

o 

d 

'd 

d 

cd 

3 

rp 


oo 

I 

e -1 vo 

£ O 


> 


. cu 

*33 d 


cd 

ft 

aj d 


cd 

Dh 


d 

'd o 
d u 

cd <-w 


'd 

43 


d 
d 
cd 

(N 

£ 7 

<U VO 

cj O 
d 

o 


d3 

44 


bo 

d M_| 

'3 S. 

& 3^ 


bX> 
d • 


44 
O , 
Dh 


£ > 


u 4 
cj w 


d 

O 

cu 

-d 


cu 

CU 

2 

_o 

cd 

d 

o 

bXi 

_d 


"cd 

Lh 

_o 

§ 

S-H 

O 

bo 

_d 


cd 

d 

43 

cu 

1 

3 

^3 

^H 

D, 

d 

Lh 

o 

a 

43 

'd 

3 

d 

X 

(U 

<u 

d 

CJ 

x> 

G 

Dh 

Q 

cd 


<U 


43 

Dh 

cd 

J-l 

cd 

Dh 

bX) 

_d 

3 

d 

cd 

C/5 


o 

.4 

Dh 


O »h 
CL, ^ 
s-, dS 
35 d 
D-, d 


bO 

'G 

,cd 

g»£ 
11 

II 

2 d 
d £ 
1/3 13 

o d 


<u 


cd i_l cd 


'd 


d 

o 

d 

cu 

_d 

td 

43 


<u 

Lh 

Dh 

cd 

Dh 

Jh 

> 

2 


d 

o 

o 

+5 

CJ 

CJ 


>s 

T3 

o 

cd 

d 

4-1 

"p 

cd 

u 

cu 

>- 

d 

'd 


cu 

^H 

> 

CO 

3 

o 

<u 


cd 

s-, 

co cd 

■S.-5 

cd cu 

5-1 bx> 

wr\ rr 1 


cd 

3 

•9 


cd 

43 


a 


O 

Dh 


d 

o 


43 

3) 

d 


a 

■s § 

<u *3 
-4 O 


<u cd 

■S a 

° o 

cd 

.. d 


Q 4 


bX3 

_d 

3 

cu 

7 

Dh 


bX) d 

d .2 


cd 


d d 43 

d 4 & 

qh Dh }_, 
cu X cd 
CO W Dh 


bXi 

_d 

+h d 
d 

I » 

cd cu 

3 S 
3 

*d 7 
d d 
cd 


3 

d 


4 < 
■a u 

cu 

CO 
CU 

'd 

I 

d Dh 

X> 3 

cd Jh 
cj cd 
O Dh 
> X5 
'd “ 


bX) 

d 

'S 

o 


bX) 

d 


Dh 

X 

CU 

d 

d 

cd 


_ bX) 
ro d 

O d - 
„ <u 
k4 1o 
Ph 43 


E % 

a 3 

d d 
C/3 o 

S & 

bp £ 
d 


o 

cd CO 

33 cu 


CU 


w d 
<u cd 
d5 - r-v 

|-§ " 
8 i G 


■ 


d 

w 

d 33 G 
<U <U bX) 

ts $ 

O .i 

Dh 


d d 

gp § 


d 44 

Cd 


LH 

'd d 


> <u 

5b Q 


I M 

fi B 

o o 


a 

d 

(U 

44 

O 

Dh 


•d Dh 

Er 7 


<U <U 

'd 'd 
cd cd 


43 

3^.2 

d cd 
ch 'd? 

g & 

<u cu 

S T3 

c 


o 

< 

u 


H-l 

pH 


cd 


o 


cd 


cd 


X! % 
H 2 

JS 

5 & 
.2 2 
13 03 
5 Dh 


s 4 

CO T3 

dJ £ 

a c 

cd cu 


43 

'd 


<U 


CU 

S-H 

Dh 

d 

o 


b* P 


cu 




d cu 

d CO 

O <u 

CO Lh 
Vh ^ 

o j3 

>> O 


O 


a B 

u ^ 

3 3 

d 2 


>> 




40 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 






M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


I 

3 

o 

Pd 


.a 

a 

a 

o 

Q 


bD 

o3 

& 

a 

o3 

1-1 


$ 


O 

Cl 


G tj 

<D 

£ £ 
«* & 


3 

-9 


o3 

-O 


CD +-* 

5-1 --- 

<D cd 

-§ | 
U O 
o3 < +h 

.. fl 


. W> 
_sd ^ 
6D bo 

c a 

<D 03 
1-1 H-I 


Q c^ 


a" 


bD 

'fl ■ 
fl 


o3 JD 
in 03 

t O H 


a 

CD 


-fl 

a 

03 

£ 

a o 

3 H-f 
& Ft 


03 o3 


S, a 

o 


o 
Cl. 

U 

fl 
Cl. Tfl 
-9 


'A O o3 


— ctf 

^ a 

^ a 

fl 


fl 

'fl 

a 

03 <N 

■ a 


| 

pq 


o 


i-H 

_fl 

CT) 

o 

<D 

'fl 

a 

CJ 

D 

s- 

a> 

fl 

03 

x 

3 

'fl 

o 

CO 

3 

« 

5 

fl 

o 

u 

O 

~a 

3 

5 


£ 

J-H 

o 

O 

<D 

o 

i-l 

3 

Ph 

3 

3 

Ph 

o 

3 


bD 




G 

o 

£ 

o 

3 

Ph 


'A 

a 


o ^ 

P4 to 

II 


P4 
o < 

Cl 


fl tj 

& a 

^ 5-. 

o3 
PC 


o3 

3 

■s 


c3 

_Q 


<d +-• 

Jh ,—I 

<U 03 

■§ 6 

U O 
o3 

.. fl 


3 § 

3) bD 

C fl 

CD 03 

1-1 H-) 


Q 3 


'fl fl 


•rt o3 
o3 .N 

a 
a 
a 

3 


ao 

# 3 

60 i 
.a jg 
=3 & 
a a 

Pi CL, 


< 

Q 


3 

13 -2 

o S 

Cl iJ 

<D to 

to _fl 

3 3 

0 § _ 
I a 00 
-2 £ 1 
"3 S >e 

CL, U U 


3 « 

03 O 

< 

Q 


4) 

3 

_fl 

a 

a 


CD 


£■ a 

a 8 

CD co 


o3 

<D 

3 

.a cn 
i ® 
£• z 


X 2P 
cd fl 


o 

Cl 

| 

'fl 

fl 


I 

■g o 

a & 
S .1 

.g 

§ t 

o I 

Zr- -3 

Si § 

11 
3 2 
3 £ 

■i .g 


< 

x y 
i s 

bD 
fl 


C 

— o 

3 & 
£ j: 

CO fl 

&, -2 
<D CD 
' ' ^ 


bD 

.fl 

3 

o3 


fl 

o 

fl 

T3 

fl 

03 


fl 

3 

u 

O 

.a 

a 


bD 

o3 

fl 

bD 

fl 

o3 


'fl 

fl 

o3 


03 OS 

fl 


60 

^ £ 


'fl 

'fl 


X 

H 


CO ___ 

Tfl o3 
03 flb 
tj <D 
'fl > 


I -O 
VO fl 

a id 


<2 bD 

.a § 

a, 3 

a ^ 
a o 
a 


.a 

a 

a 

, . 


a 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


41 






Domain 3: General Academic 


M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


P 

OJQ 

TO 

t-4 


Cl. 

a 


(D 4) 
"2 % 


(Li 

+-■ u 

TO P 3 

oj -d d 

^ .H ^ 
SL S3 u 

rr S; o) 


2 4» 

a a 


TO 


TO 

_ P 

~lj OX) 

d s 

(U TO 

H-) _) 


Sf> 4 o 


Q c* 


a 

.2 c 
00 3 -2 
fl i) td 

' & § 


TO 


_ S « 1 JS 

llfl! * 


S a .if 3 -a « 

Jc JC pq <y) 2 Oh 


Cl. 

a £ 

44 


d 3 

^ | 
o 2 
Oh .5 


- ^ 
Si) *2 
2 £ 

i ^ 


o o 

OX) o 




d 

CO ^ 

1 d 

TO 

O 15 

„ Sh 

< -3 
u 2 

d 


X 

H 


° to 

TO .+H 

S a 5 

« o a 


00 £ 
d 


Ld 'Ll 
OJ -d 

3 O 

<D O 

■s 1 

d 
o 

Cl 


<u 
d 

d <D 
O M 
u d 


— qj 

-d 3 
! d 
' d 


a a 


TO c/3 £ 
-P . - 

, *-. <D . 

i I ij 

I I 

1 d 1/3 f. 
L -rt ^ s 

! P 2 

1 2"* • 
i TO rd ■ 


d d g 
° « § ■§ 


> 

-d 


CL, C/a 


2 - 

TO" TO 
4) <u 
o > 


P ^ 

d 

O 

d 

3. > 


<u O 


d 
00 TO 


o 

d 

d 

o 


2 

2 0 3- 
•qj < | 'S 

jail 

2 d 2 52 
^ Sh P 


-d 

^ «a 

ob • y 
to 3 

S-i •-. 


OX) OX) 

P d 


Cl, 

a 


<D 4) 

2 u 


Cl, 

a 


4) 4) 

■a u 


TO 0) 

a a 


d 

x « 


44 

Sh ' 


TO (u 

a a 


3 o 


Q £2 


go ^ O 


Q c 2 


£ ob TO OX) 


CL —1 _) 


TO N 

a a 
o. a 
“ 1 


.2 

a Oh 
4) TO 


-d d 3 J? i 3 

M M oo K d 


/-, 03 

ox) o 2 
.a *p to 


2 o 


d 

d 

d 


O 

$ 


•s 3 

d >. 

d S 


d P 
•d <u 
TO 

a © 

3 CL 
OX) 

.5 O 
§ ^ 
a - 

d <D 

T3 ’S 
« P 
d ^ 


<u Cl, 

- •£ a 

11 


-O 

s S 
.a « „ 

£ s j ■ 

p to -d 
.2 « g . 
to ><; ox) 
d « <d ' 

■S g *6 

a 33 

o'd a 

, 5-, rl 
G <D M 

•P OX) —' 

00 TO j: 

P d 

TO OX) —J 
^ P 3 
<D_ _TO g 
. TO 


.3 
^ p 


d -6 


ar: 


'd h-i -d to 


OX) ' 
TO 


T3 P 

2 ’ S 
D CL 


o 

< 

Q 


_ CL 

•a a 

cc o 

CL <-> 

a 

— 1 OX) 
-d d 
P ‘3 

TO 


-d .P 

1 .a 


■> X 

^ H 


2 p 

g .2 

(A 03 

« g 
° d 
d <2 

QJ _d 


*o 


CL 

-d 

d 


& p d 

g ^ ,° 


3 p 
^ 0 ) 


d 

<u 

^ ^d 

O 2 
Z ^ 


CO $ 

a g: 
J 3 . 
fe -a 
§ 


< 

V 


H 


•X3 ^-H 
(D 00 

g o 


—■ U/ K- 

C M D 


a p 

.2 .o 


4) 


OX) i> 
to d 
va CP 


o § 
&• 'p 


j 'd _ 

d ^ 4) u 

& 5 =0 s- 
44 « 44 ^ 
«2 <2 lh 

<C to <q d 


d y 


^ & 
OX) 4) 

p p 


cl a 
d 3 


<-> d 


o ^ 
CL 


* a 3 p p 


C4 .5 CL 


4> d 
^d d 

.a o 

4) d 
44 OJ 
TO ^ 


TO 

3 a 

U* L-, 
«H 2 

3: -S 

22 4) 

3 2 

d ^ 
§ £ 
44 TO 


15 


d j-, 

P •§ 

TO TO 


-I’S 
2 S a 

- 4) TO 

151 


^ S' 

TO <4 


42 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 






Standards 

Task ELA ELP Language functions Linguistic forms of the input 


M. K. Wolf eta/. 


Building a Framework for a Next-Generation ELP Assessment System 


3 

■8 


'd 

g 


& 


o3 

G< 


o3 

X) 


X , 
o 

CLh 


W 

X5 


o 


a> 

e! 5 


bo 
_sd ^ 
b£> b£> 

c a 

<U 03 

X XI 


3 o I 


b£ 

g S 

'n 3 

03 '-G 


w < u 


o3 


O o 
<u G 
*2 O 

T) -rt ^ 

■Si"? 

a3 \0 
£ ^ O 

^ V5 

< 
P 


<u <u 


<U r* 

a ~ 

3 bO 

8 ) .3 


'd 


G ctf 

> 8 
^ G 
& 


'd 

G 

o3 


G 

.ST 

’g 

XS 


-d 

■> 


« O 




CO 

O 

U 


£ 

G 


cu 

-G 
£ 

bo -b 

.a 8 

^ G 

g 2 
<u -G 
'd G 




§> x 

8 v. 
0> 


-d 


tg 

X> 

"d 


-X G-, 

^ a 

<3 O 


'd 

G 


<u <u 

« & 

8 •§ 
o 1/5 


X3 

O 


o3 

X> 



"t. ^ 

Xh 

G-< 

X3 

U 

o3 

CJ 

H 

a 

C o 
2 8 


’bi) 

Oh „j 

o3 

(U 

'>< 

o3 


"5b 

<4 

Ph 

^ rS 
g fi) 

Lh 

bO 

03 

'-H 

o3 

bO 

o3 

G 

M) 

OJ 

X 

L-< 

o 

1 S 

OJ 

• 

§ s 

G. 

G 

o3 

• 

• 

• 

• 


Gh X 


G 

bO O 


£ o 


'd 

G 


'd cu 


•3 > 1 

3 1> VO 

S 7> O 


'd < 

bh P 


G 'G 
<u (U 


G O > 


'd 

G 


OU sO 

a p 


'd 
£ ^ 


cu 

Lh 

00 fl 

o- 

< £ 
y s 

Gh 
O Gh 

3 3 

2 | 
«s a 
a p 


G 

O 

X 

Lh 

i 


3, < 

u 


G 

O 

-O 


G 'd 
bo d 


s s 


v s ^ 
° O 


O >- 


G 

O 

X 


TJ 

G 


00 O 
I 2 
>2 <-> 
O v£ 

< G 

a -a 

M5 

W 'o 


& 


£ 3 
_L 10 
S o 

S Oh 


M _c 
<u b/D 


a >- 


s -g o s a 


o3 <-G 

a 2 


•d 


O 

G 

O 

'd 


G I 

O VO 
U 0 
<U „ 

£ C 

w? Q 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


43 


Note. ELA = English language assessment; ELP = English language proficiency. 






M. K. Wolf et al. 


Building a Framework for a Next-Generation ELP Assessment System 


4 

O 


GO 

d 

S 

cb 

oj 

CU 

C/3 


eS 

c3 


bO 

cb 

3 

GO 

3 

cb 


£ 


.3 

'cb 

a 

o 

Q 


^ a a 


cb 

3 

■9 


cb 

43 


ft 


cb 

43 


<u 


GO 
<—! cb 

GO §0 

§ § 

5-4 H-) 


LH 

3 £ 

3 O co 

*S ^ .a 

4 o b »! 


44 

O 

CU 


3 tj 
<u cb 
^ 2 
^ cb 
43 


cb 

3 

-9 


cb 

-O 


- 2 ^ - 
« & 

3 3 
bo bo 
3 3 

<u cb 
■d h-1 


3 tj 
<u ^ 
^ 2 
cb 
43 


Q P4 


<u 


.2 cu 

5-1 

3 £ 

3 3 


,cb 
U-h 
CO Jh 

GO O 
3 co 
+3 3 

<u O 

<U -J-H 


<N 

H-I 

3d 

3 


< 

o 


43 

CL, ; 


23 

O 

3d 

3 


3d 

o 


3 

_>•* , 


d( § 

aj ^ 


3d 

3 

O 

C4 

<U 

e4 


3d 

^ _ 3 a 

3 O <2 S 


C <u vo 
rJ »-h r h 


3d .. 

3 go 
cu 3 
Cu \3 

<U <U 
3d <u 
3 


GO 

3 


3d 

3 


go • 


'vi 

o 


(N 


J>^ 

> 

jw 3d 

23 3 
<u cb 

& s 

s. § 


a, -g 
<u a 

co ^ 


cb 

Cu 

3d 

3 


2d 3 

3 ~co 


3d 

3 

cb 

>. (N 

cb 7 
<U \q 

"3 O 

3 

> co 

3 OO 

° U 

3 u 


d 

go lu 
3 > 


3d 

3 


CL, 

* 


o 3 

1 I 


Cu 

X 


W 


3 

O 

+3 bo 
.3h 3 
»h ^3 

£ 

<U 

Q pC 


&• 

% 


GO 

3 


<U 


§- 


.2 cl, 
g »h 

3 3 


3 

O 

-8 

3 

3d 

3 

cb 


GO 

.3 

3 

43 


£ " 
o m 

SG J 


VO 

a 

< 

P 


3d 

H 

cb 

3 

■9 

O 
> 

"S j? 


t-s 

& 

(U 3 
_, cb 

°1 -a 

4) SR 

-5 b 


aj 3 
T3 O 
cb \3 
3 cb 
GO § 
cu H 

3 3 
3 d 


<u 


43 

CL,' 


C4 

(N 


bo 

O 


'Td 

§ ^ 

& 3 
<U cb 


r bo 

X d 

r, Cb 

O 43 
o 

Ti ^ 
L-i S-H 

'3 ° 

' <j 3 
cb O 
1 co '-3 

••H CO 
»-h <U 

^ 3 
33 ^ 

bO 

3 ^ 

W C 


3 

O 


2 vo 

-g ° 
< 
p 


3 

a" 


-3 

^3 


3 O 
cu w 

’ S cl 
& — 




cb (N 

o O 


c2 


^3 

>> 


o 

'Td 

3 

o 

C4 

CO 

a> 

34 


'Td 

3 


bO 

3 


1 




■g 

bo 


> 

cu 

'a, 

o 

gj 


o 

CO 


cu 

CU 


3d 



t 3 



3 

_o 

tu 

3 

cj 

tj 

CO 

44 

3 

3 

O 

5h 

CO 

(U 

CJ 

3 

.2 


o 

Cu 

CO 

cu 

5-1 

3d 

o 

<U 

-3 

- Sh 

<U 

CO 

3 

3d 

c3 


cb 

Lh 

Oh 

2 

cb 

.3 

-3 

Cj 

CO 

CU 

s 

b* 

3 

44 

CO 



Q 



< 


bo co 

3 « 


3 X 

2 ^ B 

cb co 

5 Ji 3 
•3 a, « 

cb 3^ J-j 


cb 

-3 


^5 - 

« & 

•5 3 

bo bo 
3 3 

<u cb 
k-I h-I 


^ cb 


Q 34 


0 ? a 

g 3 

<3 bo 

B S 

. J-H 

3 

3^ O 

u d 
3 cb 
a> 3 

S 

^ v2 

l-S 
2 > 
^ '5b 

. Sg 

’S 44 


'Td 

3 


'd 

3 


a ^ 1 

.2 cu 
T d 
b^ cu 
u ^ 


i -2 

cb 

cu 43 
44 ^ 

bO ,0 

3 i; 

^ 15 

3 -rd 


bo 

3 

3d -g 

3 3 

cb t3 

y £r 

to 3 

-3 pO 

cb g 

^ > 
'3 ^ 


X 

H 


bo 

3 


3 
•2 
a CO 

rO (U 


O 3 <! 3 4^ 


X 3 


<u 

5—( 

bo 


cu 


<u 


3d . . 
3 3d 

<U Md 

x *3 

<u 3 


o 

< 

u 


IT) 

o 


bo 

3d 

CU 

'I 

o 

3 

44 


3 3d 

■2 -5- 

I a 

II 


3" 43 43 


44 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 






M. K. Wolf ef al. 


Building a Framework for a Next-Generation ELP Assessment System 


I 

"o 

o 

-a 


.fl 

fl 

a 

o 

P 


bo 

cO 

fl 

bG 

a 

CO 

H-l 


rT* fl 


fl 

bD b£> 

fl a 

a> co 
i-3 i-3 


fl 
.o 

cO 

I 

£ O Q w 


bD O 

.B ’2 

"o 

a 

a> 
fl 
a" 

<u 


a 

CO 

bo 


< 

P 


i ■■ 

b£> 


43 

CL, 


i 

<U 

B 

o 

O 


°f> vi 
0 


3 

<£ 

'fl 

fl 

cO <N 

- O 

<u „ 

<U Jt 

is ^ 

•S U 

£ ^ 
Q a 


O 


*fl 

fl 

M 

o 

CtH 


bG ^ 
fl <d 

S cl 

3 S-H 

a< 2 
fl fl 


b£ 

fl 
cO 
43 

2 « 


< 

P 

II 

O 


<U CO 

S- g 
^ <£ 
5 fl 


fl ^ 

.2 R 

to o 
fl H-b 

JS 2; 

C4 O 
X 

<u bo 
Sh fl 

<£ a 

ky cO 
425 <u 

< a 


^ a 





■9 

O 

> 

'Hh 

fl 

co 

1 




-9 'H. 

CO f-, 

u g 

O -4h 
> ^ 




.2 


fl 

<u 

u 

<u 




(j >p 


(A 

C/3 

.2 

fl 

0 


CO 

CO 

_u 

'% to 

S3 0 


<D 

<j 

fl 

<u 

"fl 

<u 

4b 

4^ 

fl 

"fl 

43 —H 

0 2 

u fl 

cO Lh 

c 

.0 

<u 

u 

fl 

<D 

CO 

"in 

<u 

I E 

to lo 

<D 

fl 

fl 

a> 

CO 

0 

c0 

O 

S 

O 

to 

<U M-H 

1/5 ■■ 

to 

■A 

u 

c 

«U 

CO 

tj 

cO 

Uh 

O U 

s '■§ 

JZ 

0 

£ 

-2 

fl 

43 

0 

"fl 

<4 

fl 

fl 

fl a> 

O to 
u *r! 


£ 

cO 

43 

u 

flo § 

•2 S 

fl-l 

cO 

cO 

43 

<D 

be 

cO 

‘8 

h3 

cO 

»H 

O 

CO bo 

•-H fl> 

Q P4 

0 

0 

cO 

cO 

43 

<u 

be 

cO 

C r »3 
O 


43 

o 


fl 

bG bO 

C fl 

<U cO 

h-1 


P 


O ^ 

2 <» 
<U I 

43 VO 

£ o 


'fl 

fl 

O 

C4 


'fl 

fl 


§■■ 


bo 

fl 


I £ 


<* s 

^ 3 

M C4 

S G 

<U O 

fl ° 
§ 2 
15 •£ 


^ O 


< 

u 


'fl 
fl 
cO 

fl C/3 

O cO 
•fl <u 
cO 'fl 
u 

<43 -fl 

■a a 


< o 


>3 

_g 

£ 


0 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


45 






Domain 3: General Academic 


M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


46 


o 

Ph 


Ch 

d 

Ph 

<u 

bo 

cd 

d 

bD 

d 


Cl, 

3 g 


(D <L> 
3 PH 


-p s y 


CJ <u 

33 U 


d tj 
M S 


« SI 

•S 3 
to bG 

s s 

CD Cd 
i-l _J 


gf ^ o 


Q c3 


« a- 


cd 


:. lc 


SP J O 


cd 


d O 

CD H-* 

a 

8 ,9 


Q c3 


S w> 5b 
fa c c 

O CD cd 
ft hj hJ 


P $P 


a ^ 


Oh Oh GO GO 


Jg Oh 


> jy 


5 « 
ir £ 


•5 -p 


bb 3 

p 

^ rS 

T3 +-* 
^ -0 
3 


2P 


-d 

p ft 


< 

U 


oi ;d -d 


T3 O 
H cd <D 
CD C/5 

p 


p 

u 

T3 

d 
o 
, 3 
, T3 
cd 
CD 
; lh 

; T3 ro 
id U 
n *d 

n H 

8 fe 

i CD 

i § £ 

: Oh 2 


cd 

«> CD 

c 3 h 

« S 
3 "3 ! 
a oo 

2 C - 

CD O 
> ob 
,P 


a; 

pH l-P 9 h t 

3 S, S ; 
a »i ^ 
S n «; 
23 £ 3 

2 a >. 
i § a- 

S p p 


a Ph 

CD *d 
■ 53 d • 
2 ^ ■ 
-d <y 
. & « 


o 


o> VO 

.2 O 

Oh < 

2 u 


•ss -P 

l| 


<D 3 

Oi qo 


£ o 

&S 
,-S fr 

d d 

cd (D 


d p 
> ' - -2 
fc£) 13 

.a "d 

cj cd 
cd cd 
Oh C4 


-* Lh 

p -a 

cd Oh 
„ cd 
c/3 Lh 

cd cd 

£ Ph 
> Oh ^ ' 

far-p ' 

d w 

Oh C 


■d 

■d 

d 


O <£*, 


to 


■d 

d 

d 


Q £ -d 


-P d 

Oh <3 
2 8 
cd £ 
Oh <£ 

Lh d 

o — 
<u .2 

•S a 


a - 
a « 

a cd 

Lh 

J=< -P 

O Oh 
^ 2 
P i 3 

cd Oh 
<N 


d 

d 


^ 5 

O 

Oh cd 


cd cd 

a a 
£ a 
a - 

a xs 

3 3 
cd cd 


d a 

.«> j=i 

S Oh • 


Oh 

'd 

d 


(D <U 


13 | 

.a a 

,3 o 


Q oi 


■d d 

W> bX3 

p a 


d 13 


d o 


p 3 
po 2 


X " H cd 
W Q < Oh 


O ^ 

.a °? 

O vo 
Oh Q 

-d < 

P Q 
,q» S 

<D ^ 

P O 


-d 

d 

cd cd 


o 


bp fl 

.a ^ 

t3 O 

d lh 

i 3 


cd 

u 

-d w 


O ^ 


O 


< 

u 


a ^ 


s a 

Oh ^ 
<U X 


U o 

y £ 
2 


d g> 
o o^ 


o 

d 

cr 1 


-d 

> -d 


3 Ph 
£ §\ 
Ph GO 


cd d 

■ ^ EP 

5 2 

Oh O) 

3 d 
d p 
5 o 
< o 

CO 


C CD 
cd 


B S 
c S 


cd u 

<u c 
-d <d 


g -d 
2 d 
cu cd 


Sh CD 

On-Tb 

W 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 






M. K. Wolf etal. 


Building a Framework for a Next-Generation ELP Assessment System 


d 

p 

cd 

a 

o 

Q 


-P 

pH 


Ph 

a 


O) <U 

U 


2 w 

a a 


Jft 2 

S "Sd 

£ p p 

O O) cd 
ft hJ h-l 


d 

nft 

Sh 

o 

£ 5 

<D 

& 

• 

• 

• 

• 




go p 

p 2 


(< rt 

Sh u n 

£> d -H 
6 $ 

3 J £P g 

W O O U 


P Oh 


2 -P 

S ' 


o _ 

ft 8 


P o 




-PS 

CO lP <U 
cd d c* 

P O 


a 

d 

o 


' 2b 
1 t3 
! O 

I -O 

I 


O 


w ^ .2 
*H O £ 
aj h' 

> c/T < d 

Oh <D ° 
<D 00 hh 

T3 2 p 

Tl S O 


£ Ph 


g -5 

ft § 

a co 

—H (U 

*§ S 

*3 Ph 

cd P 
a pn 

P <HH 

o ° 

& 3> 


1 

& ri 

Sh CD 
Ph b 
Ph 3 
cd to 

So 

■3 -a 

2 § 


P i 

cd VO 

.b u 


a 

op p 

P cd 




O 


P d Ph 


P Hp 

<u o 

CO U 

| = 
<3 .8 

d 

p 


§ jv 

■d d 

§ S 

3 | 

£ 13 

CD Pj 

H ^3 <y 

S * 

•fti T-i 


o 


u 

u 


p p 

d o 


cd oo 
2o O 
o> 

* < 

£ y. 


CO P X 

P O - M P5 

.2 S !S M " 

P OJ U ^O 4H 

03 -T3 .-H Cj_! J_, 

3 2 O <2 

« u 


o 

p 

-p 




Id 

a 


- w > 
b s— ' cd 

<u tj d 

- g § 

T3 » 


Ph 

£ 


Ph 

a 


<u <u 

"3 pH 


ft4 

O 

Ph 

cd 


2 4» 

a a 


* » 


^ . 

6 DO 5c 
S P P 

O u cd 
ft ft ft 


P & 
O t3D d 
•p P O 


SP ^ o 


cd 


Q c4 


'n 


.2 cd 

Sh £h 


CO 2 — 

pn a S, a 

Sh "3 p O 

2 g.o U 
ft U 


P P 

a -S 


N P 

-a 3 

P 'S 

< u 


Ph ^ 

2 a 

Ph .2 

Ph d 

cd -3 

.2 ^ 

■3 d 

OJ p 


3 O 

<u 

Sh i-h 

S' Z 


'S | 

>> v2 

r, P 

<u '"2 
b "p 

P <u 

• -o 

<u 

Ph 4» 

^H - P 

Ph P <U 

i—H O d 




P 

-d 

P 


<u 


> cd 

b -5 


'd 

<u 

p 


o -d 
' P 


2 a .a 


Ph . 

P 


P 


• o ^ 
( ^ 


- p 
> 


! 13 


Ph Q Ph 


b Ph 
P Oh 

Ph CT 


iH SH CO 

cd DO P 

s ®s 

3-3 Ph _ N 

p t „r in 

o o <u b 

U q. OX) „ 

<U Ph 2 Hft 

fe- a .§ a 


O cd 

S' .2 

p -d 

Ph W 

(U S 

Hfi S 


td .2 o 

f I a 

§ IB 

js- s ^ 

.S .2 


^ .1 
2 'P 
< -3 


b - <u 

p GO oJ 


Ph <y 

p p 
o » 


> o > 

<U Sh > 
■h Ph U 
03 ^ .h 

^ -d > 

3 § o3 

2 co p 

DO O 
<y p -d 


a 

O cd 

'-b > 

3 -3 


Ph3 

--3 cd 
cd > 

S| 

Td 


■R, v „ 

2 Oh S 

<30 cd te 

3 S S 
» ft “ 

£• I 


o 

o ft 


^ M3 

co G 

o a 

< ~ 
u 


M 

s 

QJ ^ JH 

a ft 
a 2 ft 


4» (H 

p a 


2 & 

a ^p 

’ph ^2 
o ^3 
-n « 


Ph a 

Ph ^ 
cd ^3 
cd P 
Ph P 
Id "d 
2 P 


3 S' 

X) <u 

03 d 

Ti P 


p .2 
d P 

"1 P 

^ a 

^3 '3 o 

^ cd u 


Q p 


d r Ph 

P P d 

C o g 

Ph co a 

P Cd Sh 

O u ,o 

d U Vhh 

” ° Cd 

p oj o 

«» o P 

.'P ft Ph 

Sh P o 

Ph Ph "OJ 
<y oj > 

hP «3 


cd d 

a s 


ETS Research Report No. RR-14-34. © 2014 Educational Testing Service 


47 


Note. ELA = English language assessment; ELP = English language proficiency. 
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